Roe and Baker

I've been on holiday recently - yes, I flew, the first time I've gone on a foreign non-work-related trip in about a decade - so the first I heard about this was a few days ago when I bumped into someone I knew on the way home (can't go far in Boulder without meeting a climate scientist, it seems).

On the basis of "if you can't think of anything nice to say"...this ought to be a short post, but I don't have time for that, so you'll have to make do with a long one :-) RC has beaten me to it with the wonderfully diplomatic observation that the underlying idea has all been known for 20+ years but this version is "probably the most succinct and accessible treatment of the subject to date". R+B's basic point is that if "feedback" f is considered to be Gaussian, then sensitivity = l₀/(1-f) is going to be skewed, which seems fair enough. Where I part company with them is when they claim that this gives rise to some fundamental and substantial difficulty in generating more precise estimates of climate sensitivity, and also that it explains the apparent lack of progress in improving on the long-standing 1979 Charney report estimate of 1.5-4.5C at only the "likely" level. (Stoat's complaints also seem pertinent: f cannot really be a true Gaussian, unless one is willing to seriously consider large negative sensitivity, and even though a Gaussian is a widespread and often reasonable distribution, it is hard to find any theoretical or practical basis for a Gaussian abruptly truncated at 1).

Let's just recap on a small subset of the things we have observed since 1979. Most obviously, there has been about 30 years of rather steady warming, just as expected by the models at the time including most famously the Hansen prediction. The overall ocean warming is also observable, but probably a little lower than models simulate. There have been 2 major volcanic eruptions, following each of which there was a clearly observable but rather short-term cooling, exactly characteristic of a mid-range sensitivity. IIRC the magnitude and duration of the second cooling (Pinatubo) was also explicitly predicted between the eruption and the peak of the cooling itself. Perhaps most interestingly (since it does not depend either on climate models, or uncertainties in ocean heat uptake), a satellite was sent up in 1983 to measure the radiation balance of the planet, and its data since then (as analysed by Forster and Gregory last year) are in line with a low sensitivity. Of course there is a lot more we've learnt besides that, and also substantial improvements in model resolution and realism - I've just focussed some of the things that should most directly impact on estimates of climate sensitivity.

There seems to be a rather odd debate going on amongst some climate scientists about whether new observations will reduce uncertainty (I'll have more to say on this when a particular paper appears). I say it's rather odd, because I thought it was well known (it is certainly true, but true and well known are not always close cousins) that new observations are always expected to reduce uncertainty, and although it is possible that they may not do so on particular occasions, is always a surprise when this occurs. However, the vast bulk of observations (not just limited to those I have mentioned) have been singularly unexceptional, matching mid-range expectations with an uncanny accuracy (I'm ignoring stuff like ice sheets which have no direct relevance to estimating S). I fully accept that some of these observations are not be an especially stringent test of sensitivity, but they do all point the same way and it is hard to find any surprises at all in there . Remember that one of the biggest apparent surprises, the lack of warming in the satellite atmospheric record, was effectively resolved in favour of the models.

I can think of several alternative theories as to why the uncertainty in the IPCC estimate has not reduced, which R+B do not touch upon. Most obviously, I've explained (here and here) that the probabilistic methods generally used to generate these long-tailed pdfs are essentially pathological in their use of a uniform prior (under the erroneous belief that this represents "ignorance"), together with only looking at one small subset of the pertinent data at a time, and therefore do not give results that can credibly represent the opinions of informed scientists. While I think this effect probably dominates, there may also be the sociological effect of this range as some sort of anchoring device, which people are reluctant to change despite its rather shaky origins. Ramping up uncertainty (at least at the high end) is a handy lever for those who argue for strong mitigation, and it would also be naive to ignore the fact that scientists working in this area benefit from its prominence.

So in summary, Roe and Baker have now attempted to justify the pdfs that have been generated as not only reasonable, but inevitable on theoretical grounds. However, they have made no attempt to address the issues we have raised. It is notable that in their lengthy list of acknowledgees, there are many eminent and worthy scientists thanked but not one who I recognise as having actually published any work in this area - apart from Myles Allen who appears to have been a referee. The real question IMO is not whether a fat tail is inevitable, but rather whether it is possible to generate a pdf which credibly attempts to take account of the points I have raised, and still maintains any such significant tail. That challenge has remained on the table for a year and a half now, and no-one has taken it up...

Allen and Frame certainly aren't going to try, because they have gleefully seized upon Roe and Baker to justify a bait-and-switch. After failing to make any progress themselves, they have conveniently decided that it isn't such an interesting question after all, so let's not take too close a look at what has gone on thankyouverymuch. There's a couple of bizarre curve-balls in their comment: they start off by saying that the uncertainty isn't surprising because 4C warmer will be a "different planet". But nothing in Roe and Baker, or anywhere else in the relevant literature, depends on such nonlinearity in the sensitivity. In fact some of the published estimates are explicitly phrased in terms of the classical definition of a sensitivity as the derivative dT/dF (and everyone else uses this implicitly anyway). That is, the uncertainty being discussed is in our estimate of that gradient, rather than the nonlinearity as this line is extrapolated out to +3.7W/m². So I can only interpret that comment as them preparing the ground for when people eventually do get around to agreeing that the linear sensitivity is actually close to 0.75K/W/m² (~3C for doubled CO₂) so they can wring their hands and say "ooh, it might get worse in the future". Of course the reason that people use the linear sensitivity to directly derive the 2xCO₂ value is that all the evidence available, including probably every plausible model integration ever performed, indicates a modest amount of nonlinearity in that range. Allen and Frame's comment doesn't even reach the level of a hypothesis, as they have not presented any testable idea about how a significant nonlinearity could arise. There are other details I'm not very impressed by - the wording seems a bit naive and imprecise but I bet they would just say they were dumbing down for the audience so it would only seem petty to nitpick. Anyway they have at last admitted elsewhere (if grudgingly) that a uniform prior does not actually represent "no knowledge" so I see no need to pursue them further.

I don't think it is clearly expounded the R+B article itself, but in the comments to Stoat's post, Roe expounds his belief that sensitivity is intrinsically not a number, but a pdf. This seems to indicate rather muddled and confused thinking to me. True aleatory uncertainty is hard to find in the real world, and I've seen no plausible argument that the climate system exhibits it to any significant extent. We may on occasion choose to separate out some part of the uncertainty and treat it as effectively aleatory and therefore irreducible (eg consider the weather v climate distinction: if asked for the temperature on Christmas day 50 years from now, an honest answer will always be a rather broad pdf, however precisely we come to understand the forced response which will influence the shape and position of the pdf). But this is not a fundamental distinction, just a practical one - with a sufficiently accurate model and observations, the temperature really could in principle be predicted accurately. For concreteness in the current context, let's consider the following definition of S, which is based on Morgan and Keith's 1995 survey: S is defined to be the observed global temperature rise, measured as a 30-year average, 200 years after the CO₂ level is doubled from the pre-industrial level and then held fixed (with other anthropogenic forcings unchanged). This experiment is just about within mankind's grasp if we chose to do it and weren't too bothered about killing a few people along the way, so it seems to be an operationally meaningful definition (at least as a thought experiment) that would clearly result in a specific number. Repeating this experiment several times in a model with different initial conditions will give very slightly different answers, but their range will be negligibly small (< 0.1C) compared to the uncertainties in S that we are presently stuck with. The only large initial-condition-related uncertainty in model calculations of sensitivity is the well-known numerical artefact that causes some slab ocean runs to go cold, and that has no physically realistic basis. So I don't see Roe's point here to be a substantive one.

Uncanny

Quite a coincidence. On the very day after I got the email indicating acceptance of this paper, Myles Allen has a (co-authored) manuscript up on the Arxiv:

A new method for making objective probabilistic climate forecasts from numerical climate models based on Jeffreys' Prior

I thought Myles was vehemently opposed to scientists making any statements in public that had not been peer-reviewed, but maybe he was outvoted by his co-authors. Anyway, he now seems comfortable in criticising the approach of Frame et al 2005 as "arbitrary", and says that "Setting the prior to a constant [meaning uniform] is not an option". Shame he didn't agree with us three and a half years ago - or even in 2007 when he was still promoting uniform priors - but better late than never. I'm not going to gloat - seriously, I'd be glad if the whole sorry mess was finished with.

Unfortunately, it is not quite so clear that the whole sorry mess really is finally finished with. Although they now state that uniform priors are unacceptable, they don't actually go the whole hog and accept that subjective priors are unavoidable, but instead present another cook-book solution - the Jeffreys' prior! Apparently, this approach now provides an "objective" solution that eliminates the "arbitrariness". Of course Frame et al made exactly the same claims back in 2005, right down to the choice of words. Plus ça change...but this time, I suppose they really mean it :-)

As yet, it seems like no-one has actually calculated a Jeffreys' prior in any such complex case, and this paper suggests a bunch of simplifications to make it at all tractable - including the assumption that the data are independent, which of course is something Allen was quick to criticise whenever I dared to suggest it. Probably the tablets of stone are being engraved as I type and the solution will be breathlessly announced via the pages of Nature shortly.

As I said in an email recently (and demonstrated in our paper), a more constructive step IMO may not be to attempt to prescribe the one true prior that everyone one must use, but rather to check carefully what any particular prior actually means, in terms of the decisions it supports. If the prior actually reduces to "OMG we're all going to die!!11!!eleventy!1!" (as a uniform prior on S does) then we should not be overly surprised if the posterior remains somewhat alarming, even when updated with whatever data we happen to have. But so far researchers seem curiously reluctant to present their prior predictive probabilities in that way.

Uniform prior: dead at last!

As I hinted at in a previous post, I've some news regarding the uniform prior stuff. I briefly mentioned a manuscript some time ago, which at that time had only just been submitted to Climatc Change (prompted in part by Myles Allen's snarky comments, I must remember to thank him if we ever meet). Well, eventually the reviews arrived, which were basically favourable, and the paper was accepted after a few minor revisions. The final version is here, and I've pasted the abstract at the bottom of this post.

The content is essentially the same as the various rejected manuscripts we've tried to publish (eg here and here): that is, a uniform prior for climate sensitivity certainly does not represent "ignorance" and moreover is a useless convention that has no place in research that aims to be policy-relevant. With a more sensible prior (even a rather pessimistic one) there seems to be no plausible way of creating the high tails that have been a feature of most published estimates. I'm sure you can join the dots to the recent IPCC report, and the research it leant on so heavily on this topic, yourself.

Obviously there's the possibility of learning lessons about how to present criticism of existing research. This topic came up again only recently, and it's obvious that there are pros and cons to the different approaches. I saw that Gavin Schmidt published a couple of papers recently (1, 2) that were comments without being comments, in that they basically focussed on weaknesses in previous papers without explicitly being presented as "Comment on" with accompanying reply. However, I'd certainly have liked to see Allen and Frame's attempted defence appear in public, as I believe its weakness goes a long way to making our case for us. As things stand, a 3rd party reader will see our point of view but may reasonably wonder whether there are strong arguments for the other side - but don't worry, there aren't :-)

On the other hand, there is no question that the final manuscript is improved by being able to go beyond the direct remit of merely criticising a single specific paper. In particular, the simple economic analysis that we tacked on converts what might be a rather abstruse and mathematical discussion of probability into a direct statement of financial implications (albeit a rather simplified one).

I think one particular difficulty we faced with either approach is that we were not able to present a simple glib solution to the choice of prior, as we do not believe that such a solution exists. The prior that we do use (Cauchy-type) is fairly pathological and hard to recommend. In particular, if one adopts an economic analysis based on a convex utility function such as Weitzman suggests then it's not going to give sensible answers as the expected loss will always be infinite (even for 1ppm extra of CO2, essentially irrespective of what observations we make). However, that is an argument primarily in the field of economics and even philosophy, and not particularly critical as far as the climate science itself goes. The take-home message is that even with such a horrible prior, the posterior is nothing like as scary as those presented in many recent papers.

Of course, this result does bring with it my first loss in climate-related bets. Jules had wagered £500 with me that this previous paper would, if rewritten appropriately, be accepted in Climatic Change, and I was pessimistic enough to take her on. I'm quite happy to lose that bet! (I'd be happy to lose the one on 20 year trends too, if it meant that global warming was a much smaller problem than it now appears.) I suppose I should revise my opinions of the peer review system upwards a little. Apart from the extremely long delay - well over a year so far, and it's not published yet - the process worked well this time, with sensible reviewers making a number of helpful suggestions.

Anyway, here's the abstract:

The equilibrium climate response to anthropogenic forcing has long been one of the dominant, and therefore most intensively studied uncertainties, in predicting future climate change. As a result, many probabilistic estimates of the climate sensitivity (S) have been presented. In recent years, most of them have assigned significant probability to extremely high sensitivity, such as P(S > 6C) > 5%.

In this paper, we investigate some of the assumptions underlying these estimates. We show that the popular choice of a uniform prior has unacceptable properties and cannot be reasonably considered to generate meaningful and usable results. When instead reasonable assumptions are made, much greater confidence in a moderate value for S is easily justified, with an upper 95% probability limit for S easily shown to lie close to 4C, and certainly well below 6C. These results also impact strongly on projected economic losses due to climate change.

IPCC AR4 comments

At last, the IPCC AR4 is out - at least, most of it is (there is still supplementary material to come). Since I was unhappy with some of what was written in the previous (2nd) draft of Chapter 9, I looked at that first.

At first glance, I'm pleased to see that it has been significantly improved. The drafts were never meant to be a polished final version, and indeed were only released on condition that they were kept private (although the 2nd draft can easily be found on the web). So I'll restrict my comments to what they have agreed on for the final version itself.

Section 9.6 "Observational Constraints on Climate Sensitivity", contains the following:

"Note that uniform prior distributions for ECS [equilibrium climate sensitivity], which only require an expert assessment of possible range, generally assign a higher prior belief to high sensitivity than, for example, non-uniform prior distributions that depend more heavily on expert assessments (e.g., Forest et al., 2006)."

Many people may think this statement is too trivial to be worth making much of, but when I made essentially the same point about a uniform prior implying high prior belief in high sensitivity, Allen and Frame dismissed it as "just a rhetorical flourish". This statement from the IPCC also appears to directly contradict much of the peer-reviewed literature, which claims that uniform priors represent ignorance. It is encouraging to see that it is now the consensus of 2,500 climate scientists that this is not the case :-) Another significant aspect is the comment that even uniform priors "require an expert assessment of possible range", which at least takes a baby step towards acknowledging our point that the choice of upper bound can have a dramatic influence on the result. As far as I know, this critical detail (which undermines the whole rationale for uniform priors) does not appear anywhere in the peer-reviewed literature, although one reviewer did single it out as a particularly interesting point in one of our submissions. It could also conceivably be called trivial were it not for the fact that so many people have apparently been oblivious to it (or else deliberately deceptive in failing to mention it) for several years.

The defence the IPCC authors provide for the use of the uniform distribution is that it "enables comparison of constraints obtained from the data in different approaches". Of course this is not the same thing as generating a pdf which credibly represents the opinion of an intelligent researcher, but they don't actually go so far as to explicitly state this rather embarassing fact (which leads inescapably to the conclusion that these "pdfs" cannot be considered policy-relevant and used in decision support, eg economic analyses such as the Stern report etc). Most of the results they quote are based on uniform priors, but they hardly had a choice since this approach dominates the recent literature.

The section chapter also makes extensive reference to the "multiple constraints" argument (a significant feature of Hegerl et al's Nature paper, as well as our GRL paper), which is great. As I said more than a year ago, our calculation was rather simplistic and anyone who doesn't like it is welcome to generate their own answer, taking account of the arguments we have presented. Interestingly, I'm still waiting...

So in summary it might not be exactly what I would have written myself, but it's clearly a step in the right direction and it seems like the IPCC comment/review system has had some effect. We'll have to wait a little while longer to see what else they wrote about Bayesian estimation in the Appendix, since this is still not published. Whether this means Frame and Allen will now have the sense to slink away and pretend the whole sorry mess about uniform priors never really happened, remains to be seen.

Oh noes we're all going die...by 2200

I'm surprised that I'm the first to get to this, as I was trying to ignore it yesterday in the hope that someone else would blog it. Maybe no-one else reads the Independent, but the Times went behind a paywall recently so I've been shopping around.

Anyway, according to the Indescribablyoverhyped:

Almost all of the leading researchers who took part in a detailed analysis of their expert opinion believe that high levels of greenhouse gases will cause a fundamental shift in the global climate system – a tipping point – with potentially far-reaching consequences

"Almost all" means 9 out of 14, who put this probability at 90% or greater, assuming a high emissions trajectory getting to about 1000ppm CO₂ over the next 200 years. On reading the paper (here), the definition of "tipping point" seems conveniently vague. Originally a precisely-defined concept relating to hysteresis and bifurcation, it was devalued beyond all useful meaning in the Lenton et al paper, which define it as any point at which a small forcing change results in a "qualitatively different state". Without any clear definition of what "qualitatively different" means this seems more of a political construct than a scientific one.

I thought that this quote in the Indy was quite remarkable:

“We are certainly capable of committing ourselves to an emissions trajectory that make 1,000 ppm in 2200 almost inevitable if we make the wrong decisions over the next 20 years,” Dr Allen said.

I can't help thinking how incredibly fortunate we are to have worked this out just in time. Just imagine if Dr Allen had calculated that it was actually the last 20 years that were critical, and we were already committed to this dire long-term future. I suppose we would just have to party like it's 2199. Less facetiously and more directly, it seems an astonishing level of hubris that anyone (and a mere climate scientist at that) could claim to know how the next 200 years of socioeconomic development will be irrevocably (and predictably) affected by decisions we take in the next couple of decades. It would make about as much sense to claim that if only Spencer Perseval had thought more carefully he could have diverted us from our path to 393ppm today.

Despite the newspaper headline, the paper isn't really about tipping points. It's a much more wide-ranging survey of a handful of "experts who represent a range of main-stream opinion" (in the words of the paper) which follows up on a similar survey back in 1995 (Morgan and Henrion). Given the small sample of 14, I was surprised to see that no fewer than 3 (ie more than 20% of the total) were co-authors on the Stainforth et al CPDN paper which was hyped beyond all reasonable bounds (eg see here and here). And several of the rest are responsible for the silly "observationally constrained pdfs" for climate sensitivity, which as we pointed out here and here are simply pathological by construction.

Anyway, here are their new climate sensitivity estimates, presented as box and whisker plots. There is no explanation of exactly what the bars and lines mean, but the box is 25-75% and the dot is the median:
The bracketed values under each plot are the probabilities for sensitivity greater than 4.5C, so we see that 4 people put this value at 30% or greater. This seems remarkable when not a single GCM from the AR4 had such a high value, and all the decent quantitative analyses point more or less strongly to a rather lower value. Number 4 seems sane, but how anyone can claim to be certain that the sensitivity is not as low as 2.3C (number 10) seems absurd to me. Number 2 obviously has a misprint somewhere as the 25% number is incompatible with the box plot. Numbers 2, 4, 6 and 8 have extra grey plots which refer to their contributions to the previous work of Morgan and Henrion which they also participated in:

(Rotated for the sake of matching the shape.) The 4 repeat participants are Karl, Schneider, Stone and Wigley, though not necessarily in that order.

So even though as far as I can tell everyone accepts that the fundamental points we make in our two papers are valid, they still stick to these old discredited results with long tails to high values. - in fact the answers are more alarmist than 15 years ago. Makes us wonder why we bother...

The null hypothesis in climate science

Three papers have just appeared in WIREs Climate Change (here, here and here) discussing the role of the null hypothesis in climate science, especially detection and attribution.

Trenberth argues that, since the null (that we have not changed the climate) is not true, we should try to test some other null hypothesis. He sounds like someone who has just discovered that the frequentist approach is actually pretty useless in principle (as I've said many times before, it is fundamentally incapable of even addressing the questions that people want answers to), but although he seems to be grasping towards a Bayesian approach, he hasn't really got there, at least not in a coherent and clear manner. Curry is just nonsense as usual, and beside noting that she has (1) grossly misrepresented the IAC report and (2) abjectly failed to back up the claims that Curry and Webster made in a previous paper, there isn't really anything meaningful to discuss in what she said.

Myles Allen's commentary is by some distance the best of the bunch, in fact I broadly agree (shock horror) with what he has said. If one is going to take a frequentist approach, the null hypothesis of no effect is often an entirely reasonable starting point. It is important to understand that rejecting the null does not simply mean learning that there has been some effect, but it also indicates that we know (at least at some level of confidence) the direction of the effect! That is, it is not only an effect of zero which is rejected, but all possible negative (say) effects of any magnitude too - this generalisation may not be strictly correct in all possible applications of this sort of methodology, but I'm pretty sure it is true in practice for the D&A field. Especially when we are talking about the local incidence of extreme weather, there really are many cases when we have little reason for a prior belief in an anthropogenically-forced increase versus a decrease in these events, so a reasonable Bayesian approach would also start from a prior which was basically symmetric around zero. The correct interpretation of a non-rejection of the null here is not "there has been no effect" but rather "we don't know if AGW is making these events more or less likely/large". Much of Trenberth's complaint could be more productively aimed at the routine misinterpretation of D&A results, rather than the method of their generation. Trenberth also sometimes sounds like he is arguing that we should always assume that every bad thing was caused by (or at least exacerbated by) AGW, but this simply isn't tenable. Even if storminess increases in general, changes in storm tracks might lead to reduction in events in some areas, with Zahn and von Storch's work on polar lows an obvious example of this. On the other hand, there are also some types of event where we may have decent prior belief in the nature of the anthropogenically-forced change (such as temperature extremes) and in these cases it would be reasonable for a Bayesian to use a prior that reflects this belief.

I can find one thing to object to in Myles' commentary though, and that's the manner in which he tries to pre-judge the "consensus" response to Trenberth's argument. Noting that he (Allen) is in fact a major figure in forming the "consensus" in these private meetings where the handful of IPCC authors decide what to say, it sounds to me rather like a pre-emptive strike against anyone who might be tempted to take the opposing view. I would prefer it if he restricted himself to arguing on the basis of the issues rather than that he holds/forms the majority view. His behaviour here is reminiscent of the way he (and others) tried to reject our arguments about uniform priors, on the basis that everyone had already agreed that his approach was the correct solution. All that achieved was to slow the progress of knowledge by a few years.

What is probability?

I happened to come across a somewhat off-hand question "what exactly does it mean to assign probabilities for a single event?" during some random blog-surfing a few days ago. I thought it was widely accepted that such probabilities are essentially Bayesian, that is, subjective expressions of the degree of belief of a person in the proposition in question (eg as Stefan Rahmstorf writes). There are, to be sure, practical difficulties in accessing this belief in a precise and consistent manner (especially if people are prepared to lie), and personal probabilities may change from minute to minute and day to day, but the basic theory seems clear enough and forms the foundation of a large field of research with many practically useful outputs. One thing that is certainly clear (and I believe undisputed) is that the main competing interpretation (frequentism) cannot apply at all in such situations. So if you want to talk in probabilistic terms at all, you've simply got to go outside that framework, and the standard Bayesian angle seems the obvious one.

Anyway, today I finally got the Reply from Allen and Frame to our attempted Comment. [This had been accidentally omitted from the set of reviews that were sent a couple of weeks ago.] I don't intend to publish and fisk it in detail - that would be tedious, lengthy and no-one would care. However, since it was offered for publication, they can hardly complain about me making a couple of comments on it.

One striking sentence in particular jumped out at me:

"We do not think most scientists interpret probabilistic forecasts purely as expressions of degrees of belief."

(And just to clarify, the context makes it clear that this is not indended as a snide comment about the ignorance of "most scientists", but rather as support for A&F taking this same position.)

While they are being admirably clear and frank in acknowledging that they do not actually believe the estimates that they have published, it does rather raise the issue of what they consider the status of their probabilistic estimates to be.

Although I do favour what I understand to be the standard subjective Bayesian viewpoint for non-frequentist probability, I'm not dogmatically going to insist that it is the only possible one - philosophers and mathematicians have argued for centuries over probability, and I don't pretend to have all the answers or to have covered all the bases. Note, however, that Wikipedia only mentions 2 broad categories, Bayesian and Frequentist - any others seem to be rather esoteric philosophical finesses of these two, not major revolutions (excluding imprecise probability which is a whole new can of worms wholly irrelevant to this discussion). Salmon (1966) proposes three criteria for a proposed interpretion of probability:

1. Admissibility or coherence (must satisfy the Kolmogorov axioms).
2. Ascertainable (there's a method for calculating it)
   
3. Applicable (useful in real life applications)
   

Obviously, whatever A&F's interpretation is, it fails on admissibility - a point which they have also explicitly acknowledged (indeed they claim it as a feature rather than a bug). Failure on point 2 is therefore a gimme - through being multi-valued (see my previous example on P(x>4) ≠ P(x⁴>3⁴)) their methods also fail ascertainability, since any answer can be generated by reformulating the question in logically equivalent ways (hmm...I can see a semantic dodge here - does the answer "whatever you want it to be" count as a method for calculating their probability? I'll leave them to decide on that). All that needs to be shown is that their results are not useful and they'll have a 0/3 score :-) Of course this just all means that they think Salmon is wrong too, I guess...but more importantly, it leaves unanswered the question of what their version of probability actually is. What axioms does it satisfy (if any)? What does it mean?

According to their Reply (and indeed the referee who supported them), all this is entirely clear to all climate scientists (except us, I guess) and needs no further clarification. I'd be interested to hear from anyone, climate scientist or not, who can make head or tail of it!

There's a further funny point which I can't resist mentioning. Their Reply makes much of the fact that the D&A stuff (of which Myles Allen is a major contributor) routinely commits the Prosecutor's Fallacy in turning the (frequentist) confidence intervals that classical D&A methods produce, into the (Bayesian) probability intervals that people really want to see. But rather than being embarassed by this, they use it to justify their claim that a uniform prior is in fact the appropriate choice! It really is Emperor's New Clothes stuff.

On the use of the LGM to constrain climate sensitivity

Stoat posits a "challenge to JA", based on a paper by Michel Crucifix (MC). There are some slightly subtle points which require a lengthy response to do them justice, so I'll post this here rather than as a comment.

MC looks at a total of 4 models which were integrated under both LGM (Last Glacial Maximum, ~20,000 years before present) and 2xCO2 conditions, and finds little relationship between both sets of results. (He does, actually, find a very good relationship between the Antarctic cooling at the LGM and the global warming at 2xCO2 - but as he has just pointed out to me, it's an inverse relationship!) He argues on this basis that it is inappropriate to simply scale the global LGM temperature change by the ratio of 2xCO2/LGM forcings to get a value for climate sensitivity.

To be honest, at first glance I thought that it was a bit of a straw-man argument, as surely no-one is seriously suggesting that one could do such a thing. However, James Hansen (eg here which refs to here) and some others have indeed presented pretty much this argument, so in that context MC's comments seem justified. In our GRL paper, we explicitly discussed the uncertainty in the LGM/2xCO2 relationship (which we had already shown to exist here) and attempted to account for this with a dollop of additional uncertainty on top of the simple forcing calculation. No doubt there is room for debate on the details of what we did, but we were hardly blazing a speculative trail here - Myles Allen has presented a vaguely similar analysis of the LGM on p42 of this presentation, for example, and there's a similar discussion on Ch29 of the "Avoiding dangerous climate change" book, as well as the cited Hansen work etc.

The main point behind our GRL paper was no to analyse the LGM, but to point out the fallacious nature of the (implied) arguments underlying many of the published climate sensitivity estimates. IMO these are based on what amounts to rather misleading wordplay rather than a valid calculation. The argument goes roughly as follows:

If we analyse event X (and use it to update a so-called "objective" or "ignorant" uniform prior), we end up with a broad posterior pdf for sensitivity with wide bounds -> X does not provide a "useful" constraint -> we can ignore event X completely in any further calculations to estimate climate sensitivity.

The fallacy is that between those two arrows, the term "useful" has changed its meaning from "providing a tight bound on its own in conjunction with a uniform prior" to "useful at all in conjunction with other data". This erroneous argument has been variously used for both the LGM state and short-term cooling after volcanic eruptions (and possibly elsewhere). But in order for event X to be truly useless, it would have to be the case that the likelihoods P(X|S=1C), P(X|S=3C), P(X|S=6C) and P(X|S=10C) (etc) are actually all equal, and no-one has actually made this (IMO) extraordinary claim!

An unfortunate limitation of MC's work - not in any way his own fault - is that there were only 4 coupled models available with both LGM and 2xCO2 integrations at the time of his investigation, and they only covered a fairly narrow range of sensitivity, which gives little chance for a significant result to emerge (any correlation of less than 0.95 would not have been significant at the 5% threshold). I suspect that he would have found stronger results if he'd had a larger sample of models which encompassed a wider range of sensitivities (although I'm sure there would still have been uncertainty around any correlation). The Hadley Centre and/or climateprediction.net have been promising for some years now to do some ensembles of LGM simulations with their ensembles. Until they or others actually get round to it, we are pretty much twiddling our thumbs, but here is a more optimistic look at things, and there is also a recently-submitted manuscript on jules' work page. IMO the real debate is not on the binary yes/no question "Does the LGM constrain climate sensitivity?", but rather "What evidence does the LGM provide relating to climate sensitivity (and more generally, other future climate changes), and how best can we use it?" If anyone wants to argue that the answer to this is "absolutely nothing whatsoever" then they are welcome to try, but I think they will find themselves well on the scientific fringes.

Two more side-notes:

Firstly, our recent manuscript, which revisits the question of an "ignorant" or "objective" prior, does not use the LGM at all (except inasmuch as it influenced the Charney report, which I guess is not very much).

And secondly, I see that Myles Allen was quite happy to describe the work of several climate scientists as "wrong" in the presentation I linked to above. So those who accuse me of libel in my criticism of others could perhaps benefit from a sense of proportion.

Another interview!

While jules was busy getting her hair pulled over there, I was having an exclusive interview with David Rose of the Daily Mail, some of which appeared in this article (apologies to anyone who suffers an allergic reaction to any of those words - firefox users may find this add-on useful).

In the interests of openness, here is the full transcript...

The reality is that there was no interview: he never even contacted me to check he had represented my views accurately, just as he didn't ask Ed Hawkins before apparently plagiarising his graph (and misrepresenting it into the bargain).

The "increasingly untenable" quote seems to have been pulled off the Revkin article (without attribution, naturally) which quoted me recently. Andy Revkin did exchange a few emails with me to ensure he had fairly represented my view, and I have absolutely no complaint with him on that score. The bit Rose adds about "the true figure likely to be about half of the IPCC prediction in its last report in 2007" is a complete fabrication of course, it's not something I can imagine having said, or being likely. I do think the IPCC range is a bit high, expecially the 17% probability of sensitivity greater than 4.5C. But their range, or best estimate, is certainly not something I would disagree with by a factor of 2. See here for some more extensive recent commentary from me.

Given all that, it's perhaps a bit pointless to comment on the other opinions quoted by Rose, as they may also be lies. However, Piers Forster appears to defend his (IMO reasonable) comment (though not the article as a whole). I'm a little more surprised by the comment attributed to Myles Allen - if he really thinks the higher estimates are "looking iffy" then it's hard to think of anyone who could still defend them. It wasn't long ago he was arguing that the IPCC projections were too optimistic.

But they also laughed at Bozo the Clown

I'm amused to see Roger Pielke Jr playing the Galileo Gambit regarding his (Klotzbach et al) paper:

Had Michael been blogging around the time of Copernicus, he would have explained to his readers that the world is in fact flat, and that Copernicus guy must be wrong, because Michael and all of his Ptolemian friends said the world was flat, so those saying differently must be wrong because they do not jibe with his "coherence network."

Of course Roger wants to talk about politics and tribes, but I'd rather talk about science which is where this disagreement properly resides (IMO). Just to recap briefly:

Pielke and Matsui 2005 claims to have investigated the effect of temperature trends "such as due to increases in the atmospheric concentrations of the greenhouse gases, carbon dioxide and methane" on the lapse rate at night. However, they do this by applying a heat flux to the bottom boundary of the atmosphere. This is not how GHGs (or indeed any of the main climate forcings) act. Thus, their result, cited by Klotzbach et al as: "Monitoring temperature at a single height will produce a significant warm bias when the atmosphere has warmed over time [Pielke and Matsui, 2005]" is simply not valid, and there is absolutely no basis for this belief.

Roger also adds:

"Will Michael's or James' critiques of our work appear in the peer reviewed literature? Of course not (because their critiques are off target and simply wrong)."

Well, that's a hostage to fortune if ever I saw one. I will simply remark here that Myles Allen said similar things not so long ago, and I'll have a blog post or two to add on that particular topic shortly :-)

Roger Pielke Sr has also weighed in again, although I really wonder how he expects to benefit by keeping on going on about this. His persistent appeals to his own authority are somewhat undermined by his error of confusing a downwelling forcing at the land surface with a direct warming of the base of the atmosphere. His self-published email to me contains the following:

The P&M paper just looked at the issue as to whether if there was less loss of heat at night out of the top of the boundary layer, even if the loss was the same, would the vertical distribution of the heat loss be uniform between strong and windy nights?

I can't make much sense of the apparent contradiction in less loss of heat at night out of the top of the boundary layer, even if the loss was the same, but a bigger issue is why he claims to have looked at changing the heat loss at the top of the boundary layer when he clearly changed it at the bottom. The distinction is absolutely crucial, and anyone with any sort of knowledge of geophysics knows that thermally stratified fluids can behave very differently when heated from the top versus the bottom. It's a surprising error for an expert in boundary layer meteorology to make. I have emailed him directly and look forward to his explanation on this point.

As for their complaints about tone, well motes and beams come to mind. Not to mention pots and kettles.

An Inconvenient Truth

Eventually the reviews from GRL arrived for this paper (actually several days ago now, but I've been busy recently).

There were 3 reviews in all, which is unusual for a standard GRL paper.

In reverse order:

Ref 3 says the paper is "quite correct in its analysis" (he does comment on some technical details), but refrains from giving a recommendation on publication or otherwise, stating his suspicion that the points we have raised may already be accepted by the climate science community, ie the spectre of high S is just a straw man! It is hard to see how anyone who is aware of Stern's Review, and what appears to be in the IPCC draft (and numerous other papers and public comments) could really think that, but still...at least it's a clear endorsement of the principles we have presented.

Ref 2 also has a number of technical points, but recommends publication after revision and even treatment as a "GRL highlight". Interestingly, he says we are too harsh in the way we criticise the approach of Frame et al, apparently believing that they did not seriously propose to use the uniform prior U[0,20] for calculating probabilities. That's right - in his opinion, their approach is so obviously wrong that he cannot even believe they could possibly have meant such a thing - for everyone knows that there is no such thing as an "ignorant" prior.

Ref 1 is a bit of a disappointment. He doesn't seem to understand it at all, despite our attempt to explain things in such elementary terms. He is still sticking to the untenable belief that a uniform prior is "ignorant", and indeed maintains that the whole number line would be the most "uninformed" choice, even in the face of our elementary observation that a uniform prior with a wide range assigns virtual certainty to extraordinarily high sensitivity (including negative values if the prior is not truncated at 0). What makes it worse is that he's clearly an active researcher in the field. Even so, after clearly not understanding it and recommending rejection as not suitable for GRL he then strongly recommends we consider sending it to Science or Climatic Change as some sort of opinion piece!

It gets boring to point it out again, but
(a) If you don't use the probability axioms, as Allen and Frame have explicitly and repeatedly proposed, then what you are doing is simply not "probability" as the term is generally understood. This is not a matter of opinion, but a matter of definitions (at least until and unless someone proposes a new version of "probability", with some plausible basis).
(b) "The uniform prior" does not represent "ignorance" under any reasonable definition of ignorance I can think of - and no, circularly defining "ignorance" to be "the state of knowledge represented by a uniform distribution" is not reasonable!

So any attempt to present our rather elementary (and, admittedly, a bit naive) exploration of how to correctly calculate probability could hardly be suitable as some sort of opinion piece. Indeed, it would undoubtedly fall foul of referees pointing out that it is just a trivial description of probability, and no-one could seriously have ever believed otherwise...

The Editor (Chief Editor this time) obviously jumped on this "not suitable" comment and based his rejection on our paper being just an "opinion piece", telling me to send it to Nature or Science instead. You've got to laugh really - or else cry, I suppose. Couldn't he have made that judgment 10 weeks ago, rather than waiting for 3 broadly favourable reviews (even Ref 1 clearly thought it was important and publishable) and then cherry-picking the worst? In fact Jules had originally suggested Climatic Change (Ref 1's other suggestion) as a suitable destination, and we might have sent it there had GRL not explicitly suggested submitting a full 4-page paper to them. I now have a 500 quid bet with her that it will be rejected if we do send it there - a bet which I placed immediately prior to reminding her that Steven Schneider was the editor :-)

So, it is hard to see where to take it from here. After numerous reviews of various versions, it is abundantly clear that what we are saying is essentially correct - no referee has produced any significant criticism of the principles, although it is obvious that some researchers in the field simply don't understand the subject very well at all (I'm not claiming to be perfect myself, of course, but I've certainly got the gist of it). The approach of Frame et al is excused from criticism by some on the basis that it is so obviously wrong that they couldn't possibly have meant it, and the pathological pdfs that have been published and widely used in the policy debate are excused from criticism on the basis that no-one really believes them anyway. It's clear that a bunch of people are quite happy to see the Inconvenient Truth of our argument not get published. One thing that keeps me sane is that the rejections have been due primarily to journal editors rather than scientists, but the ultimate outcome is of course just the same. I guess I can go to the EGU in April and present the argument there once more, but it's pretty boring to just go and say the same obvious things again and again. Maybe, eventually, the argument that it does not need publication because everyone already knows it will actually come true. Meanwhile, people like Stern and the IPCC can only go by what is in the literature, and the Convenient Untruth of high climate sensitivity is very useful for one wing of the political debate. So I'm sure the disinformation will march on apace...

Bounds, climate sensitivity, and costs of climate change

Hot on the heels of our paper (which is still languishing in the publishing queue, though published on-line) I was rather surprised to come across another paper recently talking about upper bounds on climate sensitivity, and the costs of climate change. It is open access, so you can all read it for yourselves. The authors consider the "long tail" of possible temperature change and how this influences the economic analyses of climate change. They point out that the pathology of Weitzman's result vanishes if an upper bound on climate sensitivity is imposed. They use a Cauchy distribution for sensitivity, and show that the optimal climate policy is fairly insensitive to where this bound is placed, within the range tested of 20-50C. However, they don't appear to justify why these bounds should be used, rather than (say) 500C or 500,000C, at which point the results would probably be rather different.

Though the authors appear to not know about our Climatic Change paper, they actually do cite two of our other papers, in a way that I'm not really enthused by. They interpret us as explicitly ruling out a value for sensitivity greater than 8C, where in fact all of our results are probabilistic and do not arrive at an absolute value (other than any assumed in the prior). But this is only by way of a throwaway comment at the end of their paper, and isn't in any way central to their argument.

Coincidentally, Myles Allen and co are also going on again about how the Jeffreys' Prior solves all the problems of subjectivity (see here for previous). The whole enterprise appears to be a dead end to me and as far as I can tell, they haven't actually demonstrated any practical results, but maybe when he has eliminated all other possibilities he will reluctantly come around to embracing the standard Bayesian interpretation of probability. At least while he is presenting abstruse technical notes on the Arxiv he isn't causing more trouble elsewhere, and it must now be increasingly difficult for him to defend his previous claims. This could make life a little embarassing for the next IPCC report if people don't start producing climate sensitivity estimates that are not based on the now thoroughly discredited uniform prior...

9 1/2 weeks

Seems to be how long it takes to not get a response from GRL these days.

Yes, that means I'm still awaiting any sort of response regarding this manuscript which was submitted way back in September. For those who think I'm a bit trigger-happy to get upset about such an apparently modest time interval, note that GRL is specifically supposed to be a rapid turnover journal for short letters - a standard review request allows 2 weeks, which should give a reasonable expectation of a response in about 3 weeks including editorial handling etc.

Moreover, it's not just the rather extraordinary time delay that I'm pissed off with, but the extremely unprofessional way in which GRL seem to have handled it. Firstly, I was amazed to find out that the manuscript had been assigned to an editor who just happens to be a close colleague of Dave Frame and recent co-author with Myles Allen on a paper concerning methods for probabilistic estimation. Secondly, it's astonishing that this person didn't seem to think it was inappropriate to take on this task. And thirdly, the Chief Editor ignored my request that he should be replaced by someone without such an obvious conflict of interest. That's despite GRL actually having a box on their submission form for such editorial conflicts of interests to be mentioned - which I didn't fill in at the time of submission, as this person is nowhere listed as an editor on the GRL website (or anywhere else on the web, such as his own web-page) and I therefore had no possible reason to suspect that he, or anyone else with such an obvious relationship with those researchers who I am most directly criticising, could potentially be offered the task.

According to GRL's on-line manuscript tracking system, the reviews were all in a full 2 weeks ago and since that time have been sitting on the editor's desk waiting for him to make a decision. There has been no reply yet to the email I sent to GRL last week enquiring as to his health...

Update

No sooner blogged than I get an email from GRL...It has eventually been passed over to the Chief Editor for the decision. I'd have been happier if this had happened in advance of the (potentially critical) choicee of referees.

More sensitivity?

Myles Allen and Dave Frame would presumably be turning in their graves (were they dead), having decided some time ago that climate sensitivity was no longer interesting. However, others seem to disagree. Sherwood et al have a new paper in Nature arguing that sensitivity must be high, because only the most sensitive models have sufficient tropospheric mixing in a specific region that the authors focus on. Thanks to the several people who sent me a copy BTW.

It looks like a careful bit of work, there's no obvious problem with it, though there is one weakness that the authors are careful to mention, which is that the models may be significantly biased in some way such that an observationally-derived constraint ends in the wrong place. Eg if the models have too much mixing in some other region, or even a bias in some other process or place that compensates, then the Sherwood et al estimate will be biased. A similar issue crops up in the paleo world: if all the simulations of the Last Glacial Maximum use too little negative forcing (which we believe to be the case), then models with the correct intrinsic sensitivity will not be cold enough and a constraint based on LGM temperature change and model simulations will in that case tend to overestimate the true sensitivity.

There's not a whole lot you can do about this problem other than be aware of it, and try to use constraints that are as large scale and relevant to the predictand as possible, to reduce the risk of spurious correlations. That's one reason why I generally prefer the paleo and transient temperature change methods, as despite their own limitations they do at least directly consider the climate response to forcing on large scales. In contrast, emergent constraints generated across the GCM ensemble of climatologies might be spurious (though the ensemble size of Sherwood et al makes this seem a bit unlikely to me) or biased due to other compensating errors. Relatedly, I think I heard a rumour that the previous Fasullo and Trenberth result based on southern ocean biases in CMIP3 disappears in the CMIP5 ensemble. I might have misunderstood this, so don't quote me on it.

Road trip (part 2)

Just back from a quick trip to give two seminars in a day, first at ECMWF on the outskirts of Reading courtesy of Tim Palmer (who has also been at the Newton Institute some of the time), where a handful of us had a short time to chat about the value of short-term forecasts for assessing (IPCC-type long term) climate model performance. It's certainly an intriguing proposition and I have followed the limited work in this area (eg) with some interest. One difficulty in making progress is that the NWP people generally don't have the remit to look at long-term changes, and the climate change people don't have the remit (or indeed technical capability) to look at short term predictions.

We followed this with a quick dash to Oxford and the AOPP department at the invitation of Myles Allen. We didn't have much time to talk outside our seminar but we seem to be doing some related work where there is the possibility of mutually complementary work and perhaps ultimately collaboration. After the talk jules and I spent a bit of time talking to other people there and had a great meal in the Chiang Mai Kitchen which seems to be still going strong - it opened just before I left more years ago than I care to remember. I think this trip might be the first time I've been back to the city since I graduated so it was great to have the excuse to visit again and see a few familiar sights, if only briefly.

We certainly had fun giving the seminars themselves. At the 2nd talk, I seemed to end up abusing the concept of a truth-centred ensemble a bit more emphatically than usual, perhaps due to having on the previous day both noticed another exceedingly ropey paper appear on this topic, and also receiving this decision, from the very same journal. So I was pleased to find several members of the audience eagerly agreeing that they had never thought the truth-centred thing made much sense. Of course the people I should be talking to are the ones who are still using it as a basis for their analyses...

Oh yes, I also confirmed what I suspected here. Not that I'm losing any sleep over that, but I do think it's a strange editorial decision. And the editor concerned is unfortunately no longer with us, so I couldn't even grumble at him if I wanted to. I'm sure it will come out in the end, and I'll probably have more to say at that time.

Decadal prediction stuff part 2

Ok, having got various things out of the way, on with the show.

I liked this letter which appeared in Nature recently. Not just because I'd done something similar myself with the earlier Hansen forecast :-) In general, I think it's important to revisit historical statements to see how well they held up. Allen et al have gone back to a forecast they made about 10 years ago, and checked how well it matches up to reality. The answer is...

really well. On the left, is the original forecast with new data added, and the right is the same result re-expressed relative to a 1986-96 baseline. The forecast was originally expressed in terms of decadal means, so I don't think there is anything untoward in the smoothing. The solid line black in the left plot is the original HadCM2 output, with the dashed line and grey region representing the adjusted result after fitting to recent (at that time) obs using standard detection and attribution tchniques.

They also compared their forecast to a couple of alternative approaches:

This plot shows the HadCM2 forecast (black), CMIP5 models (blue) and another possible forecast of no forced trend, just a random walk (green). The red line is the observed temperature. They point out that their forecast performed better than the alternatives, in the sense that it assigned higher probability (density) to the observations.

So far, so good. However, I disagree with their statement that "the CMIP5 forecast also clearly outperforms the random walk, primarily because it has better sharpness" (my emphasis). Actually, the CMIP5 forecast outperforms the random walk simply because it is clearly much closer to the data. The CMIP5 mean is about 0.41 in these units (all these numbers are just read off the graph, and may not be precise), the random walk is of course 0, and the observed anomaly is 0.27. The only ways a forecast based on the CMIP5 mean could have undererformed the random walk would have been if it was either so sharp that it excluded the obs (which in practice would mean a standard deviation of 0.06 or less, resulting in a 90% range of 0.31-0.51), or so diffuse that it assiged low probability across a huge range of values (ie, a standard deviation of 0.75 or greater, with associated 90% range of -0.8 to 1.6). The actual CMIP5 width here seems to be close to 0.1, well within those sharpness bounds.

I do think I know what the authors are trying to say, which is that if you are going to be at the 5th percentile of a distribution, it's better to be at the 5th percentile of a sharp forecast than a broad one. But changing the sharpness of the forecast based on CMIP5 would obviously mean the obs were no longer at the 5th percentile! In fact, despite not quite hitting the obs, the CMIP5 forecast is not that much worse than the tuned forecast (black curve), thanks to being quite sharp. And according to the authors' own estimation of how much uncertainty they had in their original forecast, they obviously got extraordinarily lucky to hit the data so precisely. With their forecast width, it would have been almost impossible to miss the 90% interval - this would have required a very large decadal jump in temperature. I don't think it is reasonable to say that one method is intrinsically better than the other, on the basis of a single verification point that both methods actually forecast correctly. If the obs had come in at say 0.4 - which they forecast with high probability - I hardly think they would have been saying that the CMIP5 ensemble of opportunity was a superior approach.

(For what it's worth, I think the method used in this forecast intrinsically has exaggerated uncertainty, but that's another story entirely.)

Frogs and blogs

I was going to write about the frogs, but the longer I delay the more stuff comes up and now this is going to be a rather lengthy post...I'll start at the beginning and see where I end up.

Via John Fleck, I see that the wheels of science have finally turned, albeit exceeding slow...

You may recall this story from 2 years ago - someone claimed that scary global warming was killing all the poor frogs. Well, now a mere two years later, other researchers have published a paper which argues strongly that climate change is at most a minor player in this process. That seemed pretty clear right at the start, to some people at least. The lesson here is that yes, science does tend to be self-correcting, but it can be a painfully slow process, and an over-hyped story will be round the world years before the more nuanced version has got its boots on. The Pounds paper has been cited by 155, including the IPCC (uncritically). This is hardly an isolated example, either (eg see also Bryden) - John Fleck has coined the phrase "the Nature effect" for cases such as this.

It is IMO notable (and this is my second point) that the paper attracted intelligent and pertinent criticism in the blogosphere immediately after publication, well before the wheels of the peer-review process started to turn. Those two blogs I linked to are both people I've had plenty of disagreement with in the past (and probably will do in the future), but as I said at the time, that doesn't mean I will jerk my knee and reflexively dismiss anything they say, just because of who said it. Unfortunately WCR rather spoil things with their unthinking endorsement of Chylek's silly paper on paleoclimate and sensitivity - it seems their critical faculties only apply in one direction. So it's definitely a case of reader beware, but the fact remains that the truth certainly can get out there a lot faster in electronic media than through peer-reviewed publication. Of course this relies on some of the bloggers having a clue about what they are writing, which pretty much requires at least some scientists to participate.

So from the POV of my post, it is timely that Nature Geoscience has published a couple of commentaries on the value (or otherwise) for blogging, with Gavin Schmidt on one side, and Myles Allen on the other. There's discussion on RC of course. Gavin puts the case well enough - I can't imagine he found it a taxing task, as the case pretty well makes itself. Given that the internet (and blogging) exists, it is hardly credible for science as a whole to turn its back on this avenue for communication. Myles struggles to pitch blogging against the peer review system, as if they are somehow in competition. Perhaps it's too easy to just say I endorse John Fleck again. So I'll flesh out the point a little with some examples. You may recall the Schwartz paper which claimed to prove that climate sensitivity was very low. Of course it was easy to see substantial problems with the paper. Months later, our Comment is still mired in the review process (it does at least seem that it will be published eventually) but my blog posts have already been repeatedly cited in discussion about the paper, usually to rebut sceptics who are promoting it as the latest proof that global warming is a myth. (Perhaps this can be dismissed as "just the web" but one can't simultaneously argue both that no-one takes the web seriously and also that it actually undermines more traditional media.) So I see no reason to regret, or apologise for, putting the evidence out in the open before it had been peer-reviewed. I also don't feel under any obligation to follow up the rather straightforward and elementary criticism of Chylek (also here) with submitted comments, although I haven't entirely ruled out the idea. Let's not forget that peer review is hardly a faultless process - I don't necessarily disagree with Myles' assessment that it's the best system that we have, but it is well known that it is strongly biased to maintain the status quo, and routinely misses substantive errors. At its best, the careful opinions of unbiased colleagues can greatly improve the clarity and content of the paper, but the minimum threshold may simply be that two of the author's pals (yes, authors generally get to suggest some reviewers, and editors may not have the energy or expert knowledge to go outside that list) glanced over the article and didn't think that it was so bad that it was unpublishable. There is not even necessarily the implication that they think it is right, merely that they think the case is arguable. So let's not get too excited over peer review as some stamp of approval or validity (or conversely, as rejection meaning a paper is necessarily wrong). It's one piece of evidence, of uncertain strength.

As well as dredging up the old "Overselling Climate Change" thing, Myles also gets some digs in at our exchanges. I'm sort of surprised he wants to bring it up again, but I guess he couldn't resist the chance for a few snide comments. Probably I would have done the same had the boot been on the other foot :-)

Unfortunately one commentor on RC has already misinterpreted what he wrote. Where he says "needless to say, our response is not" [available to read as a rebuttal of the criticism], that should not be interpreted, as that reader did, to mean "our adversary is censoring our comments" but rather "needless to say I'd rather my responses are not made public". Of course I'd be very happy to publish the full set of exchanges here, and Dave Frame has already commented several times on my blog with no censorship - I think I even left up all of the hate mail from their colleague Carl Christensen, although I may have deleted some of his more abusive "anonymous" comments. I agree that blogs are a rather one-sided forum (usenet was better, but has been effectively destroyed by trolls) but there's nothing stopping him putting his stuff up on his own web site or the Arxiv or anywhere else he wants to. Alternatively, if he truly thinks that no non-peer-reviewed material should ever be communicated to the public, he could start by taking down the massive amount of non-peer-reviewed stuff up on the CPDN website, and their bulletin board. No, I'm not suggesting that would be a sensible step forward. but it's hard to see how else to interpret

"If, as a scientist, you feel you have to communicate non-peer-reviewed opinions to a journalist or member of the public, then stick to communicating one-to-one and make it clear you are speaking off the scientific record. Better still, don’t"

Hopefully Chris Randles will be along in a minute to explain how I'm misreading that :-)

It's pretty clear that for one reason or another journals don't like to publish comments (eg read Doswell and Errico on peer-review in general and comments in particular). Here's another anecdote I've never blogged about. It is an arithmetic error I spotted in Levitus' seminal ocean heat content paper way back in 2002. It's a just a simple error in a linear regression, so the facts are not in doubt - I was persuaded in the end (see the linked thread) to write a couple of sentences to Science pointing it out but the editor decided, aided by an extraordinary volume of bluster from the original authors, that it was not worth publishing. The data have long since been superceded but it annoyed me at the time that both Science and the authors themselves were quite happy to see a simple arithmetic error (which just coincidentally happened to exaggerate the ocean warming, and showed a rather better agreement with models than the correct analysis would have done) remain uncorrected in the literature and be used in further analyses. (Yes, looking back at that episode I now see I was rather naive at the time, having only recently shifted into climate science. I am sad to say that such behaviour would no longer strike me as unexpected.) But the idea that I should somehow be morally obliged to censor myself because an editor didn't want to devote any space to publishing that small correction is patently absurd. Equally silly is the suggestion that I should not point out the glaring errors in Chylek without feeling obliged to write a short paper. It's a shame that comments seem to be taken as such a personal insult - as I've said before, the only scientist who's never made a mistake is one who's not done any science, although I'm not suggesting that all mistakes merit a published correction (journals would be overwhelmed by trivia). But I know of two recent cases where people declined to co-author a comment due to concerns about offending the (more powerful) recipient. In both cases the potential author was untenured, and one of them explicitly cited that as the major factor in their decision. It's hardly a sign of healthy science when people are too scared to say what they think, on blogs or elsewhere.

More on multiple constraints

Hank has been pestering me to comment on this paper (preprint on Klaus Keller's web page), which appeared not so long ago. It seems to have been aimed squarely at the "we can't hope to estimate climate sensitivity" hand-wringers, and as such you won't be too surprised to find that it has my full approval. The paper illustrates how observations of surface warming, and ocean heat content, can jointly be much more powerful in climate sensitivity than either observation is by itself. The basic reason for this is elegantly demonstrated in their Figure 2: although the surface warming by itself is compatible with high sensitivity, this requires a high ocean mixing to keep the warming rate down. Ocean heat content is also by itself compatible with high sensitivity, but only if ocean mixing is low and the warming is restricted to the upper levels, otherwise the ocean takes up too much heat in total. Taking account of both the surface air temperature and the ocean heat content is only possible if sensitivity is moderate.

I expect this point must be already implicit in much previous work (eg Chris Forest and Reto Knutti have used both surface and ocean observations together) but I don't suppose it has been so clearly illustrated in this way. Note that the authors are not actually presenting an accurate estimate, but rather presenting the argument that such an estimate is in principle possible. Two reasons for this are that (1) the ocean heat observations are currently rather poor, and (2) that we don't know the total forcing accurately enough due to aerosol uncertainties. So they are restricted to presenting the hypothetical possibility of such a result, and suggesting that research to tighten up these other uncertainties would be valuable. But even so, it serves to contradict stuff like that strange Roe and Baker paper which claimed (with Frame and Allen eulogising it) that we can't possibly hope to generate an accurate estimate for fundamental physical reasons. That simply isn't true.

There's another Urban and Keller paper I am less excited by, Probabilistic hindcasts and projections of the coupled climate, carbon cycle, and Atlantic meridional overturning circulation systems: A Bayesian fusion of century-scale observations with a simple model which I think was described as "in press" at the EGU. Unfortunately it's another of these start-with-a-uniform-prior-and-don't-use-much-information papers, which therefore ends up in a moderately "alarming" result. As with the previous paper, they present it as a sort of proof of concept, so I don't think they will mind me saying that I don't find their numerical results particularly credible. Among their 18(!) parameters, one of their most critically important parameters (hydrological sensitivity) seems to be just a guess that is completely unidentifiable from the observations used. They also estimate climate sensitivity, and have generated the typical long tail with high values that is inevitable when one starts (as they did) with a uniform prior and uses little data to constrain it. Their posterior estimate for sensitivity (shown somewhere in Figure 2) appears to have a mean of about 5C and a high probability of exceeding 7C, which IMO is ridiculous (and nowhere do they explain why they disagree so violently with the entire IPCC). To his credit, Nathan Urban did say he would not use uniform priors so readily in the future, but it's too late to change this one.

Decadal prediction part 3

The first one isn't really a decadal prediction, but having finally got round to downloading the HadCRUT4 data and plotting it out, it seemed an obvious comparison to make. The pic is the famous IPCC AR4 (CMIP3) model hindcast and projections from the AR4 (it's Fig 5 in the SPM), with real data plotted on top (appropriately anomalised). I've blown up the relevant portion on the left:

As you can see, it looks pretty good up to about 2000. In fact, it looks ok up to about 2007, but since then has perhaps started to go a little bit pear-shaped. The yellow line should be ignored, of course - it's from the "constant composition" integrations, which hold the atmospheric composition fixed at year 2000 values. The other colours refer to three main projections, all of which give basically indistinguishable results over this time scale. The shaded region only covers ±1σ of the ensemble, and more importantly, I must point out quite clearly that the IPCC explicitly state that this analysis is not intended as a probabilistic prediction, so the fact that reality lies quite clearly outside this shaded areas for the last couple of years does not invalidate or falsify any particular prediction. On the other hand, it does suggest that the models are generally warming up a bit too fast. I make that statement here without prejudice as to whether this is due to errors in physics, or forcing, or just blind luck. Of course, I'm sure most of you will have also seen the equivalent with the CMIP5 models on Ed Hawkins' blog, but in that case it is more of an open question as to how much they might have been tuned to recent data. In the case of CMIP3, the "historical" scenario ended in 2000, so even if this was used to some extent (and there are strong arguments that it was), data subsequent to that is much less likely to have been used.

As Ed shows, the CMIP5 models are running a little hot too. More amusing, is to do a similar evaluation on the latest forecast from Stott et al (inc Hawkins), as published in ERL just recently. These authors did another D&A-type of analysis and concluded that "The upper end of climate model temperature projections is inconsistent with past warming". They also generated their own probabilistic predictions, based on data up to 2010. Since (in contrast to Allen et al's decadal means) their forecasts were explicitly provided for annual temp, we can already evaluate against the two years of 2011 and 2012. As Paul S pointed out in the comments, the prediction is again for decadally averaged temperatures, so we can't yet evaluate a true forecast, but we can still see how it is looking based on another 2 years of data:

Blue is HadCRUT4, showing the latest 2 years, and red is the decadal average (up to the latest available 2003-2012 interval, therefore plotted at year 2007.5). Even though they downscaled the model projections, both years since 2010 already lie outside the 5-95% range of the decadal mean prediction (though not by a huge margin). The decadal average is bumping along near the bottom of their forecast range and looks quite like likely to lie outside the CMIP5 spread, even though this is already normalised to 1986-2005.

 It's not really that surprising in hindsight, since the last year they used (2010) was an El Nino year, and yet they still managed to forecast more warming immediately following. Furthermore, their lower 5th percentile line seems to have a slope of about 0.2C per decade, implying high confidence in at least a maintenance of the recent warming rate

Incidentally, there is no sign of any El Nino on the horizon, so 2013 isn't likely to be particularly warm either (quite possibly below 2003, meaning the new decadal mean would drop). The current 2-month anomaly has it running a bit above 2012's result but still probably just outside the 5th percentile line in the above pic, though that could easily change. Anyone want to take bets on when (if?) we'll see a year inside the forecast range?

A stern review of Stern

Not really stern, more disappointed, but it was too obvious a pun to ignore. I've had a quick look at the climate science bit (the full report is here), and I don't much like what I've seen. He briefly cites our work only to ignore it - I certainly don't blame him personally for this, he can only use what climate scientists tell him and there's no doubt that our work is far from the "consensus" of the peer-reviewed publications in this area. However, it is also evident that the "consensus" is seriously flawed on this point, and it is disappointing that no-one else seems prepared to admit it or even discuss the matter in public.

From my brief glance, it seems like he uses two climate sensitivity distributions, one based on the 1.5-4.5C of Wigley and Raper (drawing on the IPCC TAR) and another higher range based on Murphy et al 2004. While he doesn't go as far as to use some of the rather silly pdfs that have been presented, he's clearly been strongly influenced by them, mentioning a 20% chance of climate sensitivity exceeding 5C a few times. Of course most of the exciting numbers being quoted from his report are those arising from the highest end of the higher range that he uses. I've said before and I'll say it again, it seems quite a hostage to fortune to base policy decisions entirely on stuff that we are all pretty confident will not happen (but merely disagree on the definition of "pretty confident").

Our work has been published for a full 6 months, and a fair number of people working in the field first saw it over a year ago, so there has been plenty of time for some sort of response (I don't necessarily mean a direct comment on it, but rather new publications which take account of the arguments we have presented here and again here). So far, we've only managed to screw out some rather limited comments from Allen and Frame, and that only through the tactic of singling them out for direct criticism. Nevertheless, they have admitted (or perhaps I should say boasted, since they seem to consider it a feature not a bug) that they do not believe the results that they themselves have generated - and note further that this admission is not merely made with reference to the particular GRL paper in question, but is a general comment on the methods they and others have widely used. One IPCC AR4 author has also admitted privately via email that he is "pretty confident that the sensitivity range is 2 to 4 K or smaller" but he's never published anything like that. Come on guys (and girls), it's time to come clean before this mess gets any worse. Just because it's in the forthcoming AR4 doesn't mean you have to defend the "consensus" to the death. At the workshop I attended this summer, someone made the (at the time) amusing comment to the effect that it would be scary what was going on in probabilistic climate prediction, were it not for the fact that it was being ignored by the politicians anyway.

Well, it's no longer being ignored.

On top of the high climate sensitivity range, Stern uses the rather extreme A2 scenario (and essentially describes it as "business as usual") for his projections, even though it is already clear even 5 years on that we are falling behind this emissions pathway. I really think it's time the economists got their act together on this. And then he adds some feedbacks on top, based on results like those of the Hadley Centre model which has an extreme Amazon dieback due to having way too little rainfall in this region even before any global warming is considered. If the Japanese model had this behaviour everyone would just say it's a crap model but because it is HADCM3 it is supposed to be alarming :-) I see RP also has some criticism of the hurricane stuff. FWIW, I don't support him 100% on his general approach (too much "its not proven" and not enough "what is a realistic estimate") but I think he's more right than wrong. Anyway, my main beef is with the probabilistic estimation, because that's what I understand best. It seems crystal clear that the methods are intrinsically faulty - indeed the errors seem rather elementary once they are stated clearly - and it is long past the time that people should have been prepared to accept this and talk about it openly. Nature's comment that our criticisms "apply more generally to a widespread methodological approach" is hardly a valid defence of the science! Stern's results appear to be heavily dependent on the small probability of extremely bad consequences, so these problems may substantially weaken the value of his report. OTOH, it might be the case that even with a climate sensitivity of 2.5C and assuming a more moderate "business as usual" emissions growth, mitigation is still amply justified (personally I think action is justifiable on a number of grounds irrespective of the supposed "climate catastrophe").

I might add some more after reading it more carefully. Or I might just let those conscientious blokes at RealClimate do it better :-)