Uncertainty on uncertainty

So Judith is going on about probability and uncertainty again, this time on the back of our "new" paper (which as you can see from its title page, was actually submitted in 2008 and previous versions of which date back a lot further). I suppose this is further evidence that the dead tree version actually means something to a lot of people. As jules suggests, I may have recalibrate my opinion about the benefits of being talked about :-)

Judith doesn't seem to like Bayesian probability. Well, that's her opinion, and it does not appear to be shared by the majority. To be clear, I don't object at all to people trying more esoteric approaches. Indeed we were quite explicit in sidestepping this debate in the paper, which does not attempt to argue that the Bayesian way is the only way. What I do object to is people throwing away the Bayesian principle on the basis of inadequate analyses. If it is to be shown inadequate, let that at least be on the basis of decent attempts.

Perhaps a useful way to think about a Bayesian analysis is that rather than magically providing us with (probabilistic) answers, it is merely a rational process to convert initial assumptions into posterior judgements: thus establishing that the posterior is only as credible as the inputs. One obvious way to test the robustness of the posterior is to try different inputs, and (subject to space constraints and the whims of reviewers) we tried to be pretty thorough in both this paper and the earlier "multiple constraints" one in GRL. People often think this just means trying different priors, but other components of the calculation are also uncertain and subjective. I've also tried to be as explicit as possible in encouraging others to present alternative calculations, rather than either blindly accepting or rejecting our own. I'm aware of a couple of reasonably current observationally-based analyses from people who were certainly aware of our arguments, and they generated estimates for the equilibrium sensitivity of ~1.9-4.7C (Forest/Sokolov group) and ~1.5-3.5C (Urban and Keller, Tellus 2010). (I read those values off their graphs, they were not explicitly presented). Like I said, it is going to be interesting to see how the IPCC handles this issue, as all these papers strongly challenge the previous consensus of the AR4.

The stuff Curry quotes at length about the lack of "accountable" forecasts (the term is a technical one) is basically a red herring. Accountable forecasts are not available for daily weather prediction either, or indeed any natural process known to man, but that does not prevent useful (and demonstrably valuable) probabilistic forecasts being made. In fact the lack of an accountable forecast system doesn't even prevent perfectly reliable (or at least arbitrarily close to perfectly reliable) probabilistic forecasts being made. What it does mean is that we need to be careful in how we make and interpret probabilistic forecasts, not least so that we don't throw out something that is actually useful, just because it does not reach a level of perfection which is actually unattainable. Which is somewhat ironic, given Judith's interpretation of what was written.

Judith summarises with "I don't know why imprecise probability paradigms aren't more widely used in climate science. Probabilities seem misleading, given the large uncertainty."

I believe the reason why these paradigms aren't more widely used because people have not yet shown that they are either necessary or sufficiently beneficial. I believe that in many areas, a sensible Bayesian analysis will generate reasonable and useful results that are adequately robust to the underlying assumptions, and I think our own sensitivity analyses, and the results I've cited above, bear this out (in the specific case of the climate sensitivity). If Judith wishes to make the case for other methods doing a better job here, she is welcome to try. In fact I've been waiting for some time for her to make a substantive contribution to back up her vague and incoherent "Italian flag" analysis. Merely handwaving about how she doesn't believe Bayesian analyses won't convince many people who actually work in this area. At least, that is my subjective opinion on the matter :-)

As a calibration of the value of her opinion, it's telling that she refers to the awful Schwartz 2007 thing as a "good paper". This was of course critically commented on not only by yours truly (along with Grant Foster, Gavin Schmidt, and Mike Mann), but perhaps more tellingly by Knutti, Frame and Allen - with whom I have not always seen eye to eye on matters of probability, so when we agree on something that may probably be taken as robust agreement! Even Nicola Scafetta found something (else) to criticise in Schwartz's analysis. Even Schwartz admitted it was wrong, in his reply to our comments! But Judith remains impressed. So much the worse for her.

I also spotted a comment on her post a couple of days ago, claiming to have found a major error in our paper. I expect Judith will answer it when she has the time, if one of her resident experts doesn't beat her to it. I'm busy with a barrel of my own fish to shoot :-)