tag:blogger.com,1999:blog-9959776.post2764621896992493456..comments2021-07-26T11:07:43.300+01:00Comments on James' Empty Blog: CoverageJames Annanhttp://www.blogger.com/profile/04318741813895533700noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-9959776.post-48989369781440349962014-05-21T14:47:30.737+01:002014-05-21T14:47:30.737+01:00Actually, I think that raises a new objection ... ...Actually, I think that raises a new objection ... S=0 or S=2 is not the same as S~N(0,1). As long as Alice picks S as advertised, the shrinkage in Bob's posterior should be OK over repeated trials.Tom Fiddamanhttps://www.blogger.com/profile/00095645518031235272noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-8631641694717127672014-05-20T22:30:59.741+01:002014-05-20T22:30:59.741+01:00Ahh ... OK, that clears it up.Ahh ... OK, that clears it up.Tom Fiddamanhttps://www.blogger.com/profile/00095645518031235272noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-54738375923914848862014-05-20T17:21:56.040+01:002014-05-20T17:21:56.040+01:00Tom, I was trying to talk about the situation of r...Tom, I was trying to talk about the situation of repeated experiments where Bob gets one obs each time, not adding new obs to the existing set. James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-75410838460314415762014-05-19T17:46:04.199+01:002014-05-19T17:46:04.199+01:00At the risk of being deemed an irrational Bayesian...At the risk of being deemed an irrational Bayesian, it seems that if Bob's estimates converge to a fixed number independent of the observations, he's not doing a very good job of updating.<br /><br />Bob's first update is from prior S~N(0,1) with O[1]~N(S,1), to N(O[1]/2,.7). But subsequent updates should be towards the observations, not toward the original prior mean 0. So Bob's next update would be to N( (O[1]/2/.5 + O[2]/1)/(.5+1), 1/sqrt(3) ).<br /><br />Bob's posterior approaches the true S after enough repetitions, with ~N((0+sum(O))/(n+1),1/sqrt(n+1)). Since all the variances are 1, this is exactly the same as augmenting a frequentist's data set with a point representing the prior, O={0,O[1],O[2]...O[n]}. There's always a residual shrinkage toward the prior, but for large n it's swamped by the data. If Alice picks 2, it approaches 2, and if Alice picks 0, it approaches 0.<br /><br />There's still a problem with coverage as a metric, because it's essentially a smoothing procedure, so it's serially correlated in any one Bob/Alice experiment. Probably better to look at something less binary, like a P value.<br /><br />Am I missing something?Tom Fiddamanhttps://www.blogger.com/profile/00095645518031235272noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-35051414000084575612014-04-22T10:52:32.559+01:002014-04-22T10:52:32.559+01:00In many cases the Jeffreys' prior is clearly v...In many cases the Jeffreys' prior is clearly very informative. The post of Nic Lewis at Climate Audit presents a perfect example of that as noted by Radford Neal and others in the discussion.<br /><br />Jeffeys' prior is rule based, but it's not uninformative.<br /><br />The information of Jeffreys' prior is dependent on the particular empirical method used, the quantities chosen to represent the empirical observations, and on the methods used to convert empirical data to the final results of the analysis. There's absolutely no fundamental reason to believe that all that would result in an uninformative prior.<br />Pekka PirilĂ¤https://www.blogger.com/profile/04747229036782463233noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-44685077478689184612014-04-22T10:13:41.731+01:002014-04-22T10:13:41.731+01:00It is obviously popular with many scientists, and ...It is obviously popular with many scientists, and in many applications it is not badly wrong (but I make no particular judgement for the specific case of carbon dating, it seems to me that a vaguely informative prior would be more plausible and not too hard to select). As for "truly uninformative", sadly that is only true (at best) using some technical definition of "uninformative" that does not correspond to common english usage. Which is more or less where we came in...James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-77878122451214154682014-04-22T09:59:39.100+01:002014-04-22T09:59:39.100+01:00The "uniform prior" is by no means the &...The "uniform prior" is by no means the "obvious choice".<br /><br />There is no obvious choice. This requires an advanced understanding of probabilistics and statistics.<br /><br />Jeffrey's Prior is the only choice that is truly uninformative.richardtolhttps://www.blogger.com/profile/14239680555557587153noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-43830032098589188282014-04-21T17:39:03.796+01:002014-04-21T17:39:03.796+01:00From Nic Lewis via email (anyone else having troub...From Nic Lewis via email (anyone else having trouble commenting on blogger?):<br /><br />===<br />James,<br /> <br />"Nic considers repetition in which the parameter is fixed and the uncertain observations are repeated."<br /> <br />No I don't. You've misread my article.<br /> <br /> <br />"id much rather trust the judgement of a scientist about plausible (prior) dates for a sample, than some automatic calculation"<br /> <br />So you think that use in OxCal of a flat prior over the whole real line represents such a judgement, and a valid one at that?<br /><br />===<br /><br />me:<br /><br />As for the first part, I'll have another look at the post, but it is abundantly clear that Nic's approach is wholly unacceptable in some cases, about which I'll post again later (holiday here).<br /><br />As for the second, I don't claim any particular expertise or experience in this area, but suspect that a uniform prior was probably selected as a somewhat lazy/naive but generally acceptable approximation.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-42945649435706716692014-04-20T11:58:19.942+01:002014-04-20T11:58:19.942+01:00Sorry, of course what was meant was in the same wa...Sorry, of course what was meant was in the same way as the Frame approach overweights the probability of surprises.EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-50852772886598565322014-04-20T11:54:04.205+01:002014-04-20T11:54:04.205+01:00A significant problem with this approach is that i...A significant problem with this approach is that it underweighs the probability of surprises in just the same way as the overbroad Frame uniform prior does especially for cases where whatever you are constructing the prior from is sparseEliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-42676739176036230212014-04-19T19:30:50.932+01:002014-04-19T19:30:50.932+01:00Thanks, I see that Radford Neal has made similar p...Thanks, I see that Radford Neal has made similar points but rather better, the comment is <a href="http://climateaudit.org/2014/04/17/radiocarbon-calibration-and-bayesian-inference/#comment-547444" rel="nofollow">here</a>. I was also going to point out that Nic's prior is nonsensical - id much rather trust the judgement of a scientist about plausible (prior) dates for a sample, than some automatic calculation that gives self-evidently ridiculous answers.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-87112391151048666492014-04-18T20:07:21.331+01:002014-04-18T20:07:21.331+01:00I am not that familiar with the term "objecti...I am not that familiar with the term "objective prior". Objectivist Bayesian I am familiar with, which just means that the interpretation of a probability is not a subjective belief, but a state of information. Thus you can have priors that incorporate information (i.e. an informative prior) without the analysis falling outside an objective Bayes framework. It would be easier if the term was used in that sense, rather than to suggest an uninformative (or minimally informative) prior is in some way more "scientifically" objective than some other. It isn't if you have information it should be included in the analysis and to choose not to is a subjective choice.<br /><br />No prior encodes no information at all, because you are at least encoding the information that you know you don't know anything about the value of some parameter. <br /><br />I am impressed that Radford Neal commented on what is a pretty obscure (from a statistical perspective) blog.Anonymoushttps://www.blogger.com/profile/16536983922816649742noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-23938373699630816152014-04-18T18:55:23.472+01:002014-04-18T18:55:23.472+01:00Kass and Wasserman have written a paper The Select...Kass and Wasserman have written a paper <i>The Selection of Prior Distributions by Formal Rules</i>. That seems a rather good description for the approach Nic Lewis has taken, but eevn that with reservations.<br /><br />Papers of Jewson, Rowlands, and Allen (2009, 2010) have also referred to objective priors. These papers contain chapters, where some derivation for the methods is presented. Going through that in detail tells that the priors are actually determined by<br /><br />- the climate model used, and<br />- the assumption that prior distributions are uniform in the space defined by variables used for representing empirical data.<br /><br />Neither of these is unique as other models lead to different priors with the same data and as making a nonlinear transformation for the empirical variables would also lead to different priors.<br />Pekka PirilĂ¤https://www.blogger.com/profile/04747229036782463233noreply@blogger.com