tag:blogger.com,1999:blog-9959776.post2801146048993578252..comments2024-02-15T04:42:41.606+00:00Comments on James' Empty Blog: Yet more on uniform priors and the misinterpretation of p-valuesJames Annanhttp://www.blogger.com/profile/04318741813895533700noreply@blogger.comBlogger45125tag:blogger.com,1999:blog-9959776.post-44396313452790043832013-03-05T18:08:26.050+00:002013-03-05T18:08:26.050+00:00There are procedures such as Delphi for handling o...There are procedures such as Delphi for handling outlier priorsEliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-91812361064985208912013-03-05T01:01:14.914+00:002013-03-05T01:01:14.914+00:00The problem is, when you get down to the details, ...The problem is, when you get down to the details, "minimally informative" doesn't actually exist.<br /><br />You give me a prior on S, I will tell you to the nth decimal place what that implies about S. Even if you thought your prior was "minimally informative".<br /><br />Chris certainly gets it.<br /><br />BTW, my Cauchy prior was intended as an extremely alarmist sensitivity test, and it demonstrates the robustness of our analysis. But if you choose a sufficiently extreme prior (such as uniform, in this case) then you can break any Bayesian analysis.<br /><br />I don't want to create some supposedly "consensus" prior because it would be too easily dismissed as just one person's opinion - and if I did a survey, what would I do with those who continue to stubbornly insist on uniform, even after I've shown why it doesn't work? I've learnt from bitter experience that people are much more likely to accept things once they have worked though it for themselves.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-57050771307995042112013-03-04T17:29:33.721+00:002013-03-04T17:29:33.721+00:00A comment from a ray @ RR
"If the prior dist...A comment from <a href="http://rabett.blogspot.com/2013/03/not-all-priors-are-equally-defensible.html" rel="nofollow">a ray @ RR</a><br /><br />"If the prior distribution, at which I am frankly guessing, has little or no effect on the result, then why bother; and if it has a large effect, then since I do not know what I am doing how would I dare act on the conclusions drawn?"--Richard Hamming<br /><br />When using a minimally informative prior, isn't Bayesian methodology really just a trick for turning likelihood into a probability? EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-59738175462258408182013-03-02T23:54:34.748+00:002013-03-02T23:54:34.748+00:00Crandles, Eli is not talking about a specific case...Crandles, Eli is not talking about a specific case, but in general about Bayesian methods. We agree that if the situation is well understood then constructing the prior will not be much of a problem. Otherwise. . . <br />EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-6578835717822140692013-03-02T10:27:25.715+00:002013-03-02T10:27:25.715+00:00>"single person [...] is a weak method.
.....>"single person [...] is a weak method.<br />...<br />construction of the prior should be structured"<br /><br />I agree that if different results can be obtained by using different reasonable priors, a multi-person derived prior would be better.<br /><br />If you don't get much different results then is there much point bothering with a structured effort at creating a prior?<br /><br />(I posed a question above of whether useful for comparing different analysis methods but no reply.)<br /><br />James maintains that there is little difference in results and certainly doesn't appear to want to make such an effort to create a multi-person prior.<br /><br />Eli, if you are not yet ready to draw the conclusion that reasonable priors make little difference, how long or for what events are you going to wait? For a paper criticising James' paper?<br /><br />If you do draw the conclusion that different reasonable priors make little difference, why are you calling for the prior to be structured? Comparing analysis methods or something else?crandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-54978501452711585702013-03-01T18:57:45.647+00:002013-03-01T18:57:45.647+00:00"That's life. How do you deal with it?&qu..."That's life. How do you deal with it?"<br /><br />Beaches and beer. <br /><br />OTOH, it is clear that a single person coming up with a prior is a weak method. Whose guru do you trust? Some sort of Delphi process (aka IPCC) would be superior.<br /><br />Eli's argument is that the construction of the prior should be structured.EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-27316275855162157512013-03-01T12:04:52.484+00:002013-03-01T12:04:52.484+00:00Eli,
>"you have no way of objectively cho...Eli,<br /><br />>"you have no way of objectively choosing the prior ... it reduces to whom do you trust which is not a very comfortable place."<br /><br />Yes different people with different subjective views will get different answers. Thats life. How do you deal with it?<br /><br />If different peoples answers are very close, great we know the answer within a reasonable uncertainty range. <br /><br />If the answers are very different then the data isn't yet good enough. <br /><br />If you still need to know what to think then you need to form a subjective opinion. Presumably you will know what period your data relates to and when it was first available. If that data is fairly recent you can look at what people thought before the time the data was available as it is difficult to see how the data affected peoples opinions before it was available.<br /><br />James did precisely this going back to Arrhenius and working forward to the start of the data used. This should give a reasonable ability to judge what a reasonable prior is. E.g. I feel the cauchy distribution James used has tails that are too fat.<br /><br />crandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-36053790806704141662013-03-01T10:32:38.176+00:002013-03-01T10:32:38.176+00:00"The problem is that the answer you get depen..."The problem is that the answer you get depends on the prior you start with and you have no way of objectively choosing the prior."<br /><br />A succinct description of the blogosphere.skankyhttps://www.blogger.com/profile/14584908320777937193noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-23803907783986263622013-03-01T03:48:23.153+00:002013-03-01T03:48:23.153+00:00One always has an ample supply of prior knowledge,...One always has an ample supply of prior knowledge, some of which may be relevant for the task at hand.<br /><br />For example, the available energy cannot be very large (as nothing explodes) nor can the inertia be very large (as nothing dashes quickly away).David B. Bensonhttps://www.blogger.com/profile/02917182411282836875noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-39843710230331030642013-03-01T03:21:29.862+00:002013-03-01T03:21:29.862+00:00The problem is that the answer you get depends on ...The problem is that the answer you get depends on the prior you start with and you have no way of objectively choosing the prior. You showed this in the example of the broad uniform prior. Eli pointed out another example. So, unless you come up with a qualifying procedure for priors it reduces to whom do you trust which is not a very comfortable place. EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-48653556180468083872013-03-01T01:40:32.933+00:002013-03-01T01:40:32.933+00:00Eli,
Agreed.
In response to your previous, there...Eli,<br /><br />Agreed.<br /><br />In response to your previous, there is no problem if the likelihood happens to be non-zero somewhere that the prior is zero - this just means you have prior knowledge that such values are impossible even though these data do not rule them out.<br /><br />If the likelihood is *only* non-zero where the prior is zero (equivalently: is zero everywhere that the prior is non-zero), then you *do* have a problem, and had better go back and work out what it was. It may not be the prior!James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-21851207502155453272013-03-01T01:02:50.124+00:002013-03-01T01:02:50.124+00:00There is no requirement that any prior have infini...There is no requirement that any prior have infinite wings.EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-57875164317547456942013-02-28T22:07:09.021+00:002013-02-28T22:07:09.021+00:00Nic, you are still confusing the issue with interp...Nic, you are still confusing the issue with interpretations of probability.<br /><br />A prior is a prior probability distribution, because this is what Bayes' Theorem and the axioms of probability require. How you interpret this probability is up to you, but the fact that it is probability is not up for debate. None of your references provide any support for you to dispute that a prior is a prior probability distribution.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-87936983608601432282013-02-28T21:13:22.174+00:002013-02-28T21:13:22.174+00:00James
You say "Prior" is simply shorthan...James<br />You say "Prior" is simply shorthand for prior probability distribution.<br /><br />But some great Bayesian statisticians regard non-informative priors being chosen as suitable to express ignorance relative to information which can be supplied by a particular experiment (Box and Tiao, 1973, p.46) or having no direct probabilistic interpretation (Bernardo and Smith, 1994, p.306). Indeed, Bernardo and Smith describes reference priors as "merely pragmatically convenient tools for the derivation of reference posterior distributions". <br /><br />Further, the great statistician who has been around longest, Don Fraser, wrote in 2011 in Default priors and approximate location models: "A prior for statistical inference can be one of three basic types: a mathematical prior originally proposed in Bayes (1763), a subjective prior presenting an opinion, or a truly objective prior based on an identifi ed frequency reference."<br /><br />So, while subjectivist Bayesians may be clear as to what they think priors represent, there are other views - just as there are as to what probability represents.<br /><br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-57347346687013033282013-02-28T11:56:12.964+00:002013-02-28T11:56:12.964+00:00Re the clueless student, I seem to recall him very...Re the clueless student, I seem to recall him very confidently predicting an ice-free summer in 2012. Steve Bloomhttps://www.blogger.com/profile/12943109973917998380noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-12965964684766961772013-02-28T11:13:39.552+00:002013-02-28T11:13:39.552+00:00Eli,
Not quite sure where you are going so hope t...Eli,<br /><br />Not quite sure where you are going so hope this isn't too irrelevant.<br /><br />The prior should still have tails outside the theoretical limit depending on how likely it is that the theoretical limit is wrong.<br /><br />If the theoretical limit is very sound, then the data may well be misleading but is still likely to indicate that the real value is close to the theoretical limit and that is what the posterior pdf will tell you. <br /><br />That might give you a tight range in your posterior when there is a chance there is a major problem with the data and the posterior should still be very wide.crandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-15464370663545506012013-02-28T05:55:03.682+00:002013-02-28T05:55:03.682+00:00To turn this argument somewhat around, consider a ...To turn this argument somewhat around, consider a case where the prior assigns a zero probability to some outcome (for example from a theoretical limit). What results if reality (aka the data) differs? EliRabetthttps://www.blogger.com/profile/07957002964638398767noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-86500383866249774192013-02-28T01:57:39.702+00:002013-02-28T01:57:39.702+00:00Chris, I'd say yes, but some random clueless s...Chris, I'd say yes, but some random clueless student probably isn't worth bothering with :-)James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-71605273182431429752013-02-28T01:56:48.030+00:002013-02-28T01:56:48.030+00:00Nic, I don't understand what you mean by "...Nic, I don't understand what you mean by "prior" if you think it does not have a probabilistic interpretation.<br /><br />"Prior" is simply shorthand for prior probability distribution.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-508830786600698322013-02-27T18:09:59.074+00:002013-02-27T18:09:59.074+00:00OT sorry. Does this sound like a suitable bet targ...OT sorry. Does this sound like a suitable bet target:<br /><br />"My projections for our planet conditions when the sea-ice has all vanished year round (PIOMAS graph projects about 2024 for this; I forecast 2020 for this) are:<br />Average global temperature: 22°C (+/- 1°C)<br />(rise of 6-8°C above present day value of about 15°C)"<br /><br />"Paul Beckwith, B.Eng, M.Sc. (Physics),<br />Ph. D. student (Climatology) and<br />Part-time Professor, University of Ottawa"<br /><br />http://arctic-news.blogspot.co.at/2012/06/when-sea-ice-is-gone.htmlcrandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-59658272467778652822013-02-27T16:15:26.315+00:002013-02-27T16:15:26.315+00:00nic-lewis,
Maybe you missed James's reply to ...nic-lewis,<br /><br />Maybe you missed James's reply to Eli saying<br /><br />"Any prior has a specific and precise probabilistic interpretation."<br /><br />So James's question did relate to a prior.<br /><br />You can stick to your view if you want, but you do seem to be unnecessarily tying yourself up in knots to try and maintain it. crandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-21494997213051863262013-02-27T14:13:48.729+00:002013-02-27T14:13:48.729+00:00James
"My proposition was very trivial and st...James<br />"My proposition was very trivial and straightforward:<br /><br />A uniform probability distribution U[0,20] for x carries with it the implication that P(x gt 6) is 70%"<br /><br />I wasn't seeking to be evasive, but I (apparently wrongly) took your question to relate to the use of a U[0,20] distribution as a prior. As I indicated, I don't think a prior has, in general, a direct probabilistic interpretation.<br /><br />Viewed simply as a probability distribution for x, then indeed U[0,20] implies P(x gt 6) is 70%. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-31342515021277860572013-02-27T07:54:11.412+00:002013-02-27T07:54:11.412+00:00Agree that James argument in the paper for experts...Agree that James argument in the paper for experts is a good one... Just do not see it as a definitive one. A bell shaped narrow distribution might be strange I do not know at the moment how that might play out and perhaps there are other reasons not to have a to low upper limit... <br /><br />Sure permafrost is not effecting sensitivity per definition but the reason for all of this more or less is for economical models? And for that it will have more or less the same implications? (I also mentions other what ifs) So using experts might narrow the upper limit to much... no way to know for sure. However, got to admit that James puts forward a strong argument and that it seams better then the other (however have not seen literature on it and I am new to all of this I confess, and there are other problems with economical models). <br /><br />Just looking at the examples in the paper linked I am a bit surprised at how much you could narrow the span with expert guessing... with the use of single studies. Still a good argument but no prof.Magnushttps://www.blogger.com/profile/01617272924116099306noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-78011194371227902872013-02-27T07:17:16.378+00:002013-02-27T07:17:16.378+00:00Chris seems to have covered it all very well, than...Chris seems to have covered it all very well, thanks!<br /><br />Nic, your reply seems rather evasive and awkward to me. There is no reason to introduce all sorts of quibbles and conditions about likelihoods. My proposition was very trivial and straightforward:<br /><br />A uniform probability distribution U[0,20] for x carries with it the implication that P(x gt 6) is 70%.<br /><br />If you cannot agree unequivocally with this statement, unconditionally and irrespective of how you might interpret and apply the concept of "probability" in the real world, then you simply aren't talking about probability.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-3082428696164999792013-02-26T23:36:54.266+00:002013-02-26T23:36:54.266+00:00Magnus,
Yes it follows that there is a lower limi...Magnus,<br /><br />Yes it follows that there is a lower limit that will get a sensible answer. But that is likely to be a case of two wrongs happening to make a right. The cliff edges of a uniform prior are unlikely to be sensible, far more likely the probabilities of a sensible prior tail away to very low levels smoothly.<br /><br />Permafrost of course releases GHGs. So this shouldn't affect climate sensitivity (i.e. temperature change in response to a doubling of CO2 levels.) Instead it is an extra GHG forcing.<br /><br />Sensitivity may change but the likely direction is downward as sea and other ice disappears.<br /><br />(To avoid giving the wrong impression, I think there is plenty of reason for more action to reduce GHG emissions. More risk would make more action appropriate. We don't need to falsely play up the risks to make action appropriate.)crandleshttps://www.blogger.com/profile/15181530527401007161noreply@blogger.com