tag:blogger.com,1999:blog-9959776.post113783499986582137..comments2019-09-06T01:25:53.102+01:00Comments on James' Empty Blog: Probability, prediction and verification IV: More on Bayesian vs frequentist uncertaintyJames Annanhttp://www.blogger.com/profile/04318741813895533700noreply@blogger.comBlogger26125tag:blogger.com,1999:blog-9959776.post-65170488026717134382007-02-07T20:13:00.000+00:002007-02-07T20:13:00.000+00:00Take a statistics class. If you don't know a numb...Take a statistics class. If you don't know a number is prime or not, there is still a knowable probability . Its better than nothing, thats why statisticians get paid. They add value.<br /><br />As for weather, I was taught that 70% meant that for the whole area, 70% of the area would definitely get rain at least once. Your mileage may vary. But since you are in meteorology, you ought to have a definition at least as precise as that.Greghttps://www.blogger.com/profile/08923908304768825538noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138770127161048032006-02-01T05:02:00.000+00:002006-02-01T05:02:00.000+00:00One more on subjective certainty, and then I will ...One more on subjective certainty, and then I will shut up. <BR/><BR/>I heard a girl on National Public Radio (US) say that she knew she was right in disbelieving evolution because she had her "Lord Jesus in her heart." No way can I match her degree of subjective certainty in my belief that Darwin is right. Jesus has never even broached the subject with me, so all I have is some facts and a rather long chain of inference. So from a Bayesian standpoint, it looks to me like Darwin is losing - which is why the Bayesian viewpoint doesn't impress me.CapitalistImperialistPighttps://www.blogger.com/profile/06112308711915925900noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138769624104360522006-02-01T04:53:00.000+00:002006-02-01T04:53:00.000+00:00It's possible that I don't really understand what ...<I>It's possible that I don't really understand what you mean by Bayesian or frequentist</I><BR/><BR/>I suspect that many people fall into this category. They are taught via the frequentist paradigm, and instinctively try to reject the "subjective" nature of Bayesianism, but in fact they apply this concept widely in everyday life (including scientific research). It's a sort of interesting doublethink that many people are hardly aware of and reluctant to acknowledge. A longer post will be forthcoming when I have the time.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138769152142971032006-02-01T04:45:00.000+00:002006-02-01T04:45:00.000+00:00James - It's possible that I don't really understa...James - It's possible that I don't really understand what you mean by Bayesian or frequentist, but I'm pretty sure that I understand what I mean when I say meaningful probabilities come from a theory that gives frequencies.<BR/><BR/>I didn't think I was restricted to tomorrow's rainfall forecast, but even conceding that Navier-Stokes are deterministic, nobody thinks they are a fundamental description of the dynamics of a real fluid - they are just a large scale approximation, even without quantum effects. In any case, even if the equations were exact, they are only one piece of the description of the atmosphere - the other piece is the initial and boundary conditions, which can never be known exactly.<BR/><BR/>Finally, when a 70% rain forecast is made around here anyway, it often means that rain will fall on about 70% of the local area - theory predicts raincloud density in a very frequentist manner.CapitalistImperialistPighttps://www.blogger.com/profile/06112308711915925900noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138766888404906592006-02-01T04:08:00.000+00:002006-02-01T04:08:00.000+00:00Sorry CIP, but you are completely wrong about the ...Sorry CIP, but you are completely wrong about the atmosphere. For the purposes of tomorrow's forecast, it is definitely appropriate to consider it as a deterministic (but imperfectly known) system. There's no way that any quantum indeterminacy can have a measurable effect on this time scale. We do occasionally put in randomness as a proxy for our ignorance, but this is very clearly a Bayesian approach. No-one actually thinks that the Navier-stokes equations are <I>really</I> random!<BR/><BR/>If, when faced with epistemic uncertainty, you are prepared to invent hypothetical nonexistent frequentist thought experiments with a specified (how?) prior distribution from which reality is considered to be a sample, then you are merely a Bayesian in disguise. And if you are neither an honest Bayesian, or a disguised one, but instead a strict frequentist, then you'll have trouble ever making any decisions at all, rational or not.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138763998884359132006-02-01T03:19:00.000+00:002006-02-01T03:19:00.000+00:00Er, "nown" should be "known."Er, "nown" should be "known."CapitalistImperialistPighttps://www.blogger.com/profile/06112308711915925900noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138763835783008052006-02-01T03:17:00.000+00:002006-02-01T03:17:00.000+00:00Sorry James, but I don't buy it. The Earth's atmo...Sorry James, but I don't buy it. The Earth's atmosphere is not a deterministic system in any meaningful sense. Even if all initial conditions were nown perfectly, quantum effects induce an essential indeterminacy at the micro level which propagates up to every scale via the "butterfly effect." More immediately, it isn't possible to know the initial conditions very accurately at all. The atmosphere is much better modelled as a semi-deterministic system with large random elements.<BR/><BR/>Back to your ten digit composite integer = 17x73x994817. All the arguments about primality were based on frequency considerations. It seems to me that all such estimates are either frequentist in origin or pure prejudices. A further rant on the subject is <A HREF="http://capitalistimperialistpig.blogspot.com/2006/01/bayes-bogus.html" REL="nofollow">Bayes Bogus? here</A>.CapitalistImperialistPighttps://www.blogger.com/profile/06112308711915925900noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138630209702718262006-01-30T14:10:00.000+00:002006-01-30T14:10:00.000+00:00We threw the die but it is concealed from view and...We threw the die but it is concealed from view and we do not yet know the answer. However, we took high speed video footage of the first quarter second of the throw. This was provided to two teams along with similar footage of 1000 similar dice roll where the answer is known and provided to all teams.<BR/><BR/>The first team gains some knowledge from the video footage and testing on the 1000 rolls allows them to estimate a 25% probability of a six. The second team is more skilled and their testing enables them to predict a 50% probability of a six.<BR/><BR/>You and I are members of a third team. We don't get the video footage to analyse but we do get to analyse the results of the other two teams for the 1000 rolls and the one roll. <BR/><BR/>It is possible that all the information gained by the first team is fully utilised by the second team and we will be able to do no better than copy the second teams prediction. <BR/><BR/>It is also possible however that some information gained by the first team is not fully utilised by the second team, so our approach may enable us to do better than the second team. In this case, where both the first and second teams both predict an increased chance of a six, we may be able to predict a six with a 51% probability.<BR/><BR/>So what is the correct probability of that one particular dice throw being a 6? Is it 1 in 6? or is it 25%? or is it 50%? or is it 51%?<BR/><BR/>It seems to me that the answer depends on how much information you have access to. Team 3 should always win the competition or at least jointly win at getting the best prediction (though that prediction could turn out wrong).<BR/><BR/>Is the approach that I am suggesting for our third team unscientific? I see no reason to think so.<BR/><BR/>crandlesAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138584496663605322006-01-30T01:28:00.000+00:002006-01-30T01:28:00.000+00:00Last anon comment:No, your modifications are not v...Last anon comment:<BR/><BR/>No, your modifications are not valid. Try this for size:<BR/><BR/><I>An analogy with number theory may be helpful. The probability of throwing an odd number with a fair die with spots of 1,2,3,4,5,6 is half. That's a perfectly good frequentist statement. {We threw the die but <B>it is concealed from view and we do not yet know the answer</B>} ....But what about roll of this die? Does it make sense to talk about this number being odd with probability half? I suspect that some, perhaps most, number theorists would be uneasy about subscribing to that statement. Any particular number is either odd, or not. This fact may be currently unknown to me and you, but it is not random in a frequentist sense. How ever many times you look at this die you will always get the same result, whether it be "odd" or "even".</I><BR/><BR/>It should, I think, be immediately apparent that the only difference between my version and yours is the ignorance of the observer as to the state of the die.<BR/><BR/>I can see I'm going to have to go over this one more time...James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138575025620409102006-01-29T22:50:00.000+00:002006-01-29T22:50:00.000+00:00James - I have a post commenting on yours here.James - I have a post commenting on yours <A HREF="http://capitalistimperialistpig.blogspot.com/2006/01/bayesian-at-moon.html" REL="nofollow">here</A>.CapitalistImperialistPighttps://www.blogger.com/profile/06112308711915925900noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138555609633229402006-01-29T17:26:00.000+00:002006-01-29T17:26:00.000+00:00The original, with some modifications:An analogy w...The original, with some modifications:<BR/><BR/>An analogy with number theory may be helpful. The probability of throwing an odd number with a fair dice with spots of 1,2,3,4,5,6 is half. That's a perfectly good frequentist statement. {We threw the dice and got 4. } ....But what about 4? Does it make sense to talk about this number being odd with probability half? I suspect that some, perhaps most, number theorists would be uneasy about subscribing to that statement. Any particular number is either odd, or not. This fact may be currently unknown to me and you, but it is not random in a frequentist sense. Testing a number will always give the same result, whether it be "odd" or "even".<BR/><BR/>I think one can see that the prime number stuff is merely a distraction.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138552628052497592006-01-29T16:37:00.000+00:002006-01-29T16:37:00.000+00:00It should also be clear that "70% chance of rain o...<I>It should also be clear that "70% chance of rain on Jan 25 2006" and "70% chance of rain on Jan 25 2003" are both in principle equally valid statements. I won't know whether rain fell on the 25th Jan 2006 for a couple of days, but it would probably take even longer to find out about 25th Jan 2003 (even assuming someone has kept a record for the location in question). </I><BR/><BR/>If the newspaper published a random ten digit number each day, and you wanted to know whether the one published on Jan 25, 2003 is prime or not, the same type of estimation holds good, even though we're talking about a definite number.<BR/><BR/>Do explain how one would be led to seek a better predictor in the Bayesian interpretation of probabilities.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138552068997051182006-01-29T16:27:00.000+00:002006-01-29T16:27:00.000+00:00You didn't pick a number randomly; but the predict...You didn't pick a number randomly; but the predictor didn't know that. You're trying to cram two different situations into one. In the case of the weather (assuming it is deterministic) and in the case of "pick a number, is it prime?", the probabilistic prediction arises out of ignorance. When you dispell the ignorance, by providing more information, by actually being able to do the computation, or by actually witnessing the weather event, the probabilities change. <BR/><BR/>This is no more surprising than a predictor which said the chances of getting a 5 on a throw of a dice is 1 in 6, and then you throw the dice and now it is something definite; and to say that the probability of that particular throw being a 5 is 1/6 when a 3 is staring at you in the face seems absurd. Likewise, having picked a composite number, to give it a probability of being a prime seems absurd. <BR/><BR/>Ultimately, if there is more information to be had (e.g., instead of picking numbers randomly, you're always picking a composite number), the failure of the predictor (in a frequentist sense) will be what prompts you to look for a better predictor.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138545474463245792006-01-29T14:37:00.000+00:002006-01-29T14:37:00.000+00:00Perhaps I should explain myself a bit better.In ph...Perhaps I should explain myself a bit better.<BR/><BR/>In physics statements about probability makes sense in two cases:<BR/>i) quantum theory <BR/>ii) systems where we do not know the initial state exactly.<BR/><BR/>In all cases we must be confident about the theory (kinematics and dynamic), otherwise the use of probability is misleading.<BR/><BR/>Probability is not a good replacement for missing or wrong theories. Your example of the prime number illustrates this very well.<BR/><BR/>In the more interesting case of climate change:<BR/>Bayesian probabilities cannot and should not be used to replace a true understanding of how the various components and forcings etc. interact, IMHO.Wolfganghttp://wbtsm.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138545106188207182006-01-29T14:31:00.000+00:002006-01-29T14:31:00.000+00:00James,> The atmosphere is a deterministicyes, but ...James,<BR/><BR/>> The atmosphere is a deterministic<BR/>yes, but we do not know the initial conditions exactly.<BR/>Thus, different scenarios are compatible with our knowledge of the current state of the atmosphere.Wolfganghttp://wbtsm.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138542816430043452006-01-29T13:53:00.000+00:002006-01-29T13:53:00.000+00:00In the same way, "it will rain tomorrow" is either...In the same way, "it will rain tomorrow" is either true or false. The atmosphere is a deterministic system!James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138542345940611392006-01-29T13:45:00.000+00:002006-01-29T13:45:00.000+00:00> How would you apply this sort of reasoning to es...> How would you apply this sort of reasoning to estimating the primality of 1,234,567,897<BR/><BR/>I would not. There can only be one scenario in this case. Mathematical statements are either true or false.Wolfganghttp://wbtsm.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138519628640365442006-01-29T07:27:00.000+00:002006-01-29T07:27:00.000+00:00But I didn't pick that number randomly, and we do ...But I didn't pick that number randomly, and we do know its value, so it is not a random input. It doesn't seem to me that your proposed interpretation of the situation really brings any insight, and it is rather tortured in situations where there isn't a sensible frequentist "wrapper" to dress the question up in. There are a number of different priors that could have produced that number, which have strongly different probabilities of generating a prime in a frequentist experiment. Maybe I selected it from the 10-digit composites, for instance! Or the 10-digit primes plus 1,234,567,897.James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138513301564439142006-01-29T05:41:00.000+00:002006-01-29T05:41:00.000+00:00How would you apply this sort of reasoning to esti...<I>How would you apply this sort of reasoning to estimating the primality of 1,234,567,897 - "we have 100 different versions of 1,234,567,897, and according to our current knowledge, 4 of them are prime and 96 are composite"?</I><BR/><BR/>Almost. Given our state of knowledge - a process that is randomly picking numbers with a uniform probabiliity in a particular range - 4 out of every 100 picked are predicted to be prime.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138495050162808992006-01-29T00:37:00.000+00:002006-01-29T00:37:00.000+00:00Wolfgang,How would you apply this sort of reasonin...Wolfgang,<BR/><BR/>How would you apply this sort of reasoning to estimating the primality of 1,234,567,897 - "we have 100 different versions of 1,234,567,897, and according to our current knowledge, 4 of them are prime and 96 are composite"?James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138471817351225322006-01-28T18:10:00.000+00:002006-01-28T18:10:00.000+00:00In my opinion, the only way to interpret a stateme...In my opinion, the only way to interpret a statement like the "70% chance of rain tomorrow" is:<BR/><BR/>We have 100 different scenarios (or simulations), all compatibel with our current knowledge of the atmosphere, and 70 of those show rain for tomorrow.<BR/><BR/>In my opinion, counting the number of scenarios (or simulations, forecats etc.) is the correct way to understand bayesian probability as a frequentist.Wolfganghttp://wbtsm.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138426889781552572006-01-28T05:41:00.000+00:002006-01-28T05:41:00.000+00:00Anon,With modern ensemble-based methods, the basic...Anon,<BR/><BR/>With modern ensemble-based methods, the basic principles of making a probabilistic forecast are easy enough to explain. Eg, a forecast centre can run 10 forecasts with a numerical atmosphere model, each one starting off from a slightly different representation of today's atmospheric conditions (which are not known with certainty anyway). If 7 of the 10 produce rain on the following day at the location in question, then they say "70% chance of rain tomorrow". In practice it is a bit more complicated than that. Some comments on forecast verification (your 2nd comment) may be coming to a blog near you soon...<BR/><BR/>One of Roger's references can be found <A HREF="http://ams.allenpress.com/amsonline/?request=get-abstract&doi=10.1175%2F1520-0477(1980)061%3C0695:MOPPF%3E2.0.CO%3B2" REL="nofollow">here</A>, by the way ("Print version" gives the full pdf).James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138374429268476902006-01-27T15:07:00.000+00:002006-01-27T15:07:00.000+00:00Alan Murphy published a couple of studies on this ...Alan Murphy published a couple of studies on this -- <BR/><BR/>On the misinterpretation of precipitation probability forecasts. Bulletin of the American Meteorological Society, 58, 1297-1299, 1977.<BR/><BR/>Misinterpretations of precipitation probability forecasts. Bulletin of the American Meteorological Society, 61, 695-701, 1980. (A.H. Murphy, S. Lichtenstein, B. Fischhoff, and R.L. Winkler)Roger Pielke, Jr.noreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138305117194123122006-01-26T19:51:00.000+00:002006-01-26T19:51:00.000+00:00A slightly different question from "what do they m...A slightly different question from "what do they mean by 70% chance of rain?" would be "how do they argue for a 70% chance of rain?". A Bayesian interpretation gets you past the meaning question, but I don't think it helps much with the justification. Do forecasters use statistical databases, or what? Going along with this, there's a slightly different way to interpret the question "what's the right probability?", which is "how does the community of weather forecasters decide if a forecast is good or bad?" I mean, before whatever weather is going to happen actually happens, just looking at the weather forecast, what criteria are there by which one can distinguish a "good" one from a "bad" one?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9959776.post-1138248881056871312006-01-26T04:14:00.000+00:002006-01-26T04:14:00.000+00:00I think that a 70% rain forecast can in principle ...I think that a 70% rain forecast can in principle mean either that showers will hit about 70% of the area, or that a whole area of solid rain cloud is approaching but will only arrive (fall) wth 70% probability.<BR/><BR/>From the individual end-user's perspective (and simple analyses of rainfall/forecast statistics etc), they are pretty much the same, although there could be some differences especially for those interested in larger areas. Crop failure in 5% of the world would be very different proposition from a 5% chance of complete failure across the whole world, for example!James Annanhttps://www.blogger.com/profile/04318741813895533700noreply@blogger.com