tag:blogger.com,1999:blog-9959776.post6968738422001286230..comments2024-02-15T04:42:41.606+00:00Comments on James' Empty Blog: Objective probability or automatic nonsense?James Annanhttp://www.blogger.com/profile/04318741813895533700noreply@blogger.comBlogger35125tag:blogger.com,1999:blog-9959776.post-42992319283852453542014-05-21T19:21:02.893+01:002014-05-21T19:21:02.893+01:00Regarding "one thing that Nic's method wi...Regarding "one thing that Nic's method will say with certainty is "this is not a <br /><br />Kamakura-era artefact"! ... Nic's posterior (green solid curve) is flatlining along the <br /><br />axis over the range 650-900y, meaning zero probability for this whole range. The obvious <br /><br />reason for this is that his prior (dashed line) is also flatlining here, making it <br /><br />essentially impossible for any evidence, no matter how strong, to overturn the prior <br /><br />presumption that the age is not in this range."<br /><br />I think this is actually an incorrect statement of what the figure shows.<br /><br />Statements like "x is in this range" are about the CDF, not the pdf. Nic's posterior pdf <br /><br />here says that there's a low probability that an artifact lies in any _individual_ year <br /><br />between 650 and 900. That's a natural consequence of the fact that the chunk of C14 <br /><br />density that maps to that era is spread over a long calendar age, due to the degenerate <br /><br />transfer function. But it's not the same as saying the era is excluded; if you look at the <br /><br />CDF, its 90% or 95% range would certainly include the era.<br /><br />Nic's posterior matches what you'd get if you generate a random sample from the Normal C14 <br /><br />age distribution and pass it through the calibration curve, which seems like a pretty <br /><br />intuitive procedure to me. Of course, this begs the question of why you'd bother <br /><br />constructing an arcane prior in calendar space when it's much easier in C14 space, which <br /><br />he actually discusses in the text.<br /><br />Even though Nic's argument strikes me as technically correct as far as it goes, the <br /><br />trimodal prior on calendar age is intuitively repugnant. So, I went back to the Bronk <br /><br />Ramsey methods paper.<br /><br />I think Nic has actually missed an absolutely crucial point. In his zeal for an <br /><br />uninformative p(calendar age), he skipped over BR's explanation, "This prior reflects that <br /><br />fact that some 14C measurements (notably those on the plateaus) are more likely than <br /><br />others." In other words, BR is making the auxiliary assumption that there is a constant <br /><br />rate artifact-generating process.<br /><br />The distribution of c14 age measurements arising from a constant artifact-generating process is such that the likelihood of observing a given measurement is highest exactly where the Fisher information from the transform (i.e. the slope) is lowest. The two effects cancel, rendering the uniform prior for calendar age sensible. Score one for intuition.<br /><br />This means that, if you present Nic's method with a random selection of artifacts from the real world, it'll perform poorly.<br /><br />You could still argue about the true or least informative distribution for artifacts - I would guess that there's actually a survival bias that renders it nonuniform, but rather gently.Tom Fiddamanhttps://www.blogger.com/profile/00095645518031235272noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-62344510198218414002014-04-25T16:35:57.245+01:002014-04-25T16:35:57.245+01:00Horrors, all agree. Time for another blogger ethi...Horrors, all agree. Time for another blogger ethics panelEliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-51676516375065098472014-04-25T16:31:32.681+01:002014-04-25T16:31:32.681+01:00Crandles,
Your conclusion is correct. The consequ...Crandles,<br /><br />Your conclusion is correct. The consequences of that problem are probably the reason that the confidence limits given for calendar date of samples listed on page 10 of <a href="http://www3.nd.edu/~nsl/Lectures/phys178/pdf/chap3_3.pdf" rel="nofollow">this presentation</a> have very often upper or lower limits in the ranges 350-420 BC and 750-830 BC, but in no case in the range 510-750 BC and in only two cases in the range 420-510 BC.<br /><br />The method is very weak in resolving dates 400-750 BC, therefore those dates occur rarely in as end points of confidence intervals, but the confidence interval includes often this whole range.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-44450457306000712972014-04-25T15:32:37.384+01:002014-04-25T15:32:37.384+01:00Perhaps I am reading it wrong, but to me it seems ...Perhaps I am reading it wrong, but to me it seems that the calibration curve is saying that carbon-14 dating is a hopeless method if you want to determine whether an item is 700-800 years old or just a bit younger or older as these give identical results.<br /><br />So, if you are already confident your item is 600-900 years old and you do use such a method then prior and posterior should be very similar.<br /><br />It seems to me there are more outcomes than just this item is now more likely (less likely) to be 700-800 years old.<br /><br />This example brings out the conclusion that this method is not very good at telling.<br /><br />It seems using more than one prior remains sensible to help get at other possibilities rather than aiming for one invariant prior.crandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-15504526750256851712014-04-25T07:27:03.623+01:002014-04-25T07:27:03.623+01:00Eli,
Thanks for the link.
In my latest comment I ...Eli,<br />Thanks for the link.<br /><br />In my latest comment I tried to explain very briefly, how the Bayesian approach can be extended more explicitly to the handling of all the types of uncertainties those slides tell about.<br /><br />You are surely right that all users of the method are not equally competent. Errors are made in doing the preparatory steps, and in interpreting the results. <br /><br />There's hardly any field of science, where statistical methods are always used correctly. Well prepared standard software is of great help in that. The proposals of Kennan and Lewis would represent a really serious step in a wrong direction, but there might also be much potential for improvement. That's the part I cannot say anything more that to consider it likely, because errors are so common all around.Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-17329311806315149932014-04-25T00:26:17.064+01:002014-04-25T00:26:17.064+01:00Pekka, take a look at this presentation esp down t...Pekka, take a <a href="http://www3.nd.edu/~nsl/Lectures/phys178/pdf/chap3_1_2.pdf" rel="nofollow">look at this presentation</a> esp down to the bottom. The preparation of samples looks as if it requires a great deal of care and expertise, and in <a href="http://rabett.blogspot.com/2007/04/found-in-margins-recently-eli-has-been.html" rel="nofollow">Eli's experience</a> that varies all over the place. <br /><br />Captcha CERVECERIA. Eli wins the weekEliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-36322758403675161652014-04-24T08:58:04.415+01:002014-04-24T08:58:04.415+01:00Coming back to the question of Eli, I had some mor...Coming back to the question of Eli, I had some more ideas about an answer.<br /><br />In the Bayesian approach, what is determined from the experiment are likelihoods. That's what the experiment tells, pdf's require also a prior.<br /><br />What the experiment really tells is one single number, an integer that's the count recorded by the detector and the counter. As such this is a precise number with no uncertainty. All uncertainties can be handled elsewhere.<br /><br />Now we wish to calculate the likelihoods that exactly this number is observed for all the values of the variables we are interested in. In this case we have only a single variable, the calendar age.<br /><br />Calculating the likelihoods consists of following steps:<br /><br />1) Determine the amount of carbon in the sample, both of the age being studied and possible contamination from other times and the efficiency of the detector in observing C14 decays taking into account geometry and all other factors. Present all this information as pdf's (assuming Gaussian distributions is probably justified).<br /><br />2) Determine from the calibration data (the band with data on its shape) the probabilities of each C14 age given the precise calendar age. Do that also for the ages of possible contamination.<br /><br />3) For each C14 age calculate the pdf of counts taking into account the uncertainties of step one. If the uncertainties are small the distribution is a Poisson distribution, with corrections it could be a convolution of Gaussian and Poisson. Again do that also for the C14 ages of potential contamination. Take into account also effects like the time between the growth of a tree and manufacturing of the sample being studied and other comparable factors.<br /><br />4) Combine the results of steps (2) and (3) to get the probability of the actually observed count. These probabilities form the relative likelihoods of each calendar date. <br /><br />(The probabilities of all counts add up to one for each calendar age. Picking from the same set of numbers the probabilities of a single value of count for every calendar age results in relative likelihoods that do not add up to one, and should not be summed at all without the addition of a prior.)<br /><br />Up to this point there should not be much disagreement. We have converted all relevant data to a set of likelihoods. Doing that we have extracted all the information the measurement can tell about the calendar date.<br /><br />As a result we have an unnormalized likelihood function that tells in relative terms, how strongly the measurement favors some calendar ages over others. To give confidence intervals or full pdf's we must add a prior. It makes absolutely no sense to determine the prior based on the empirical setup. How we perform the measurement has no bearing on the probability of various ages of the sample. The prior must be fixed by some other arguments. It could be uniform in calendar time or it could be inversely proportional to the age, or we might use some additional information pertinent to that specific sample. That's up to the person who interprets the results. The measurement can tell only the relative likelihoods.<br /><br />In steps (1), (2), and (3) pdf's of contributing factors are used. They are real probabilities that describe some effectively random contributions to the expected count for a given calendar age.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-26668072776285814132014-04-23T23:23:51.656+01:002014-04-23T23:23:51.656+01:00Rabbit, the source of the width is explained in Ke...Rabbit, the source of the width is explained in Keenan(2012), referring to Stuiver and Polach(1977)<br /><br />(A Poisson radiation process leading to logGaussian, approximated to Gaussian. The measurement protocol can probably further be fine tuned for age of the samples etc. Also the width can be further reduced to practically zero if you have the money to pay for that)Anonymoushttps://www.blogger.com/profile/07089417868500194484noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-60544553796418090182014-04-23T19:58:14.174+01:002014-04-23T19:58:14.174+01:00Eli,
What you write sounds reasonable, but for me...Eli,<br /><br />What you write sounds reasonable, but for me to say something more specific would require digging deep in the procedures.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-73718429610978466042014-04-23T18:45:09.338+01:002014-04-23T18:45:09.338+01:00Pekka,
The width of the pink Gaussian on the side...Pekka,<br /><br />The width of the pink Gaussian on the side of the figures seems, at least to Eli, to be a by gosh and golly seat of the pants estimate handed down by a guru of all of the different sources of variation in the counting experiment, but not just for a particular sample.<br /><br />Finding a real error budget has proven difficult, and it appears that most people just use what pops out of a program that all the C14 folk use. There is no doubt that that curve is totally frequentest, and as such, (you were the first to mention this?) the place where Bayseans could help is finding a better way to estimate that width.EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-66848228018853482012014-04-23T16:14:45.323+01:002014-04-23T16:14:45.323+01:00Eli,
(You are often too concise and cryptic for m...Eli,<br /><br />(You are often too concise and cryptic for me. That may be, in part, due to cultural differences.)<br /><br />That's a part of the problem that I have just accepted as correct enough presentation of all uncertainties related to the sample and the actual data collection.<br /><br />By "correct enough" in the above I mean that the correct distribution is roughly that wide in comparison to the width of the bluish band and its wiggles.<br /><br />Little of what I have written is dependent even on that, but there might be something that is.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-79981513806849348262014-04-23T14:57:27.356+01:002014-04-23T14:57:27.356+01:00Prekka, Eli's question is what is the source o...Prekka, Eli's question is what is the source of the width of the pink "Gaussian"EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-89502274581271336832014-04-23T14:28:03.876+01:002014-04-23T14:28:03.876+01:00Eli,
What I mean can be explained by this figure ...Eli,<br /><br />What I mean can be explained by <a href="http://climateaudit.files.wordpress.com/2014/04/nicl_radioc1_keenan2012_fig1oxcal.png" rel="nofollow">this figure</a> from Nic's post and further from Keenan (2012) according to Nic.<br /><br />In that case the red distribution tells about the uncertainty in the determination of the C14 value, and the width of the narrow bluish band about the uncertainty in going from a given C14 value to real date. (The wiggles of the band add another aspect of uncertainty.)<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-6597461744910375762014-04-23T12:11:57.117+01:002014-04-23T12:11:57.117+01:00Prekka, what do you mean by uncertainty in the C14...Prekka, what do you mean by uncertainty in the C14 value? There are several steps from specimen to C14 content and then more in going from C14 content to calendar age.EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-56070778336801062052014-04-23T09:52:43.515+01:002014-04-23T09:52:43.515+01:00As far as I have understood, the uncertainty in th...As far as I have understood, the uncertainty in the calibration curve has been analyzed and taken into account in the software used in interpreting C14 values. That uncertainty seems, however, to be very small in comparison with the uncertainty in the determination of the C14 value, and therefore a very minor factor in these considerations.<br /><br />The main original source in the uncertainty is in the determination of the C14 value, but that uncertainty may be amplified greatly by the unfortunate form of the calibration curve (the real one, not only the artificial one shown in this post).<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-49047023347158471832014-04-23T08:16:32.720+01:002014-04-23T08:16:32.720+01:00This example clarifies very well just what is wron...This example clarifies very well just what is wrong with unthinking application of the Jeffrey's prior. In what way does the true age of the objects being analyzed depend on the calibration curve? It doesn't, there is no good reason to suppose that more objects made 2000-1700, about 1000 and 500-0 years ago are more likely to be subjected to analysis than objects produced during the intervening periods, so why should the prior favour those intervals? It shouldn't. The uniform prior seems much more in accord with what we actually don't know.<br /><br />While a Jeffrey's prior may have mathematical rigour, that doesn't mean we shouldn't ignore common sense, or check the consequences of the mathematically rigorous prior.<br /><br />I suspect part of the problem would go away if the uncertainty in estimating the calibration curve were taken into account.Anonymoushttps://www.blogger.com/profile/16536983922816649742noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-10172389252132371332014-04-23T08:05:23.347+01:002014-04-23T08:05:23.347+01:00Richard S. Tol wrote: "There is nothing intui...Richard S. Tol wrote: "There is nothing intuitive about statistics and probability."<br /><br />I disagree, the Bayesian conception of probability is straightforward and intuitive (but the analysis is often mathematically taxing), the problem with "statistics" is that it is generally performed in a frequentist setting, where the analysis is easy, but the definition of a probability is deeply unintuitive. Ideally both sets of tools should be in our toolbox though.<br /><br />"Indeed, the hard part of teaching statistics is to erase sloppy intuition and replace it with rigorous mathematics."<br /><br />No, if you do that you only mask the sloppy intuition with a veneer of mathematics, but the sloppy thinking is still there. To apply probability and statistics correctly you need your intuition to be correct as well. Much better to fix the sloppy intuition and then reinforce it with mathematical rigour.<br /><br />"So, indeed "uninformative" does not refer to the common English usage of the word "information". Instead, "uninformative" means "proportional to the square root of the determinant of the Fisher information", where Fisher Info is the accepted, rigorous definition of information."<br /><br />This is the sort of thing that results from being overly concerned with mathematical rigour rather than common sense (i.e. intuition). Why describe Jeffreys' prior as "uninformative" (which it isn't) rather than "invariant" (which it is in a fairly general sense)? What is gained by this?<br /><br />If you want to make probability and statistics intuitive then needlessly using terminology in a way that is not in accordance with everyday usage seems like a bit of an own goal.<br /><br />Don't misunderstand me, I like mathematical rigour, but I also like engineering common sense, and the two are not mutually exclusive.Anonymoushttps://www.blogger.com/profile/16536983922816649742noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-79748720198220979632014-04-22T22:54:22.718+01:002014-04-22T22:54:22.718+01:00Those contrived statistical machinations are of a ...Those contrived statistical machinations are of a minor concern compared to the (unreported) measurement uncertainty in the calibration curve.<br /><br />That's the largest source of error! and its not even reported. Sloppy.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-36354518129902125552014-04-22T22:06:08.741+01:002014-04-22T22:06:08.741+01:00Eli,
A lot of information is used to create the c...Eli,<br /><br />A lot of information is used to create the calibration curve, i.e., to determine the relationship between measured C14 and the true age. When that relationship has been determined, the likelihoods of different true ages that correspond to a particular measured C14 value can be determined. Up to that point of the analysis no priors are used.<br /><br />Priors enter if and when the likelihood values are used to determine posterior probabilities (a pdf or confidence intervals). Thus it's possible to perform the whole empirical work and present it's results in full without any reference to any prior. The limitation of that is that the results are not probabilities but (relative) likelihoods. The likelihood curve need not integrate to one. Actually it's not appropriate to calculate any integrals of it, because that would lead to a confusion with probabilities.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-63063278777444707832014-04-22T21:47:35.292+01:002014-04-22T21:47:35.292+01:00Prekka, the stylized curve that Lewis uses hides a...Prekka, the stylized curve that Lewis uses hides a lot. It forces a strong peak in the prior where the wiggle is in the center. It would do so where there is any wiggle, and indeed, because there are lots of wiggles in the real calibration data, you have to use metadata to get sensible answers and build sensible priors.<br /><br />http://www.geo.arizona.edu/palynology/geos462/10radiometric.html EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-51330717181647752102014-04-22T20:50:42.685+01:002014-04-22T20:50:42.685+01:00That part of my comment was purely semantic, i.e. ...That part of my comment was purely semantic, i.e. about understanding the meaning of the expression.<br /><br />I don't think we have any disagreement on the content (on this point at least), I was just proposing a different way of using words. It may well be that my proposal was not a good one.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-70507794459304447562014-04-22T20:40:22.312+01:002014-04-22T20:40:22.312+01:00Pekka: The likelihood on the carbon date is fine. ...Pekka: The likelihood on the carbon date is fine. The likelihood on the calendar date is degenerate. The latter matters.<br /><br />A degenerate likelihood implies a degenerate prior -- as it should -- and together they make a degenerate posterior -- as it should.<br /><br />Fix the transfer function and all is fine.<br /><br />Leave the prior out of it.richardtolhttps://www.blogger.com/profile/14239680555557587153noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-22323188068417025722014-04-22T18:55:39.290+01:002014-04-22T18:55:39.290+01:00Isn't the problem rather a degenerate transfer...Isn't the problem rather a degenerate transfer function than a degenerate likelihood.<br /><br />The likelihood derived from the empirical results is well defined. It's constant over wide ranges, i.e. it cannot differentiate at all between values in these ranges, but that doesn't lead to any technical difficulties or in any problems in interpreting directly the likelihoods.<br /><br />All problems are related to the prior and they are serious specifically, when an attempt is made to use the Jeffreys' prior. That leads to an obviously nonsensical PDF.<br /><br />Due to the flat likelihood function, any choice of prior has a strong influence. That may be considered a problem, but it's a problem of priors.<br /><br />Another way of describing the situation is that radiacarbon dating is useless as a tool for telling what the true value is within the range 500-950 years or in the range 1050-1700 years, and of limited value also in making a difference between these two ranges. This idealized radiocarbon dating is useful only for dates more recent than 400 years or in the range 1750-2000 years. <br /><br />This is weakness of the method. It's not uncommon that various methods have good resolving power only in a limited range of the quantity to be determined.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-91326548181600233212014-04-22T18:27:08.255+01:002014-04-22T18:27:08.255+01:00While the discussion here and elsewhere focuses on...While the discussion here and elsewhere focuses on choice of prior, the problem at hand is a degenerate likelihood.<br /><br />Using a prior to fix a likelihood is a bad idea.richardtolhttps://www.blogger.com/profile/14239680555557587153noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-82579893975362235202014-04-22T17:19:13.213+01:002014-04-22T17:19:13.213+01:00I don't dare to give an opinion on, whether an...I don't dare to give an opinion on, whether any prior can be described as the square root of the determinant of the Fischer information, but many and very different ones certainly can.<br /><br />The theory of Haar measures might give the answer for that question, but I know very little about Haar measures.<br /><br />To me it's more significant that Jeffreys' prior may lead to nonsensical results. Being maximally noninformative with respect to some measure is not a useful property, if the measure is pathological, and I would call a measure that corresponds to the C14 values pathological, when the issue being studied is the determination of the true age of a sample.<br />Pekka Pirilähttps://www.blogger.com/profile/04747229036782463233noreply@blogger.com