Sunday, February 08, 2009

Hansen's El Nino forecast reprised

I have no idea why Roger Pielke chose to bring up James Hansen's old 2006 El Nino forecast again just recently, but since he did, and it generated some hot air in the blogosphere, I might as well add my view.

1. Hansen originally said in a draft paper:
We suggest that an El Nino is likely to originate in 2006 and that there is a good chance it will be a “super El Nino”, rivaling the 1983 and 1997-1998 El Ninos, which were successively labeled the “El Nino of the century” as they were of unprecedented strength in the previous 100 years.
[quote taken from Prometheus, I don't have Hansen's draft myself]. The meaning of "likely" is usually interpreted in the range of 66-90% probability, so clearly the "good chance" of a super El Nino, which is conditional on there being an El Nino at all, is lower than this. Given the historical context of what a "super El Nino" is (there have been 2 in the past 30 years by Hansen's definition) a "good chance" cannot reasonably be interpreted at above 50% probability, and may have been meant as something substantially lower (albeit higher than climatology). Incidentally, I am not inclined, as some are, to give Hansen a free pass on that "super El Nino" prediction on the basis that it did not make it to further versions of the paper - it was clearly his original opinion, despite the fact that he withdrew it in the face of opposing views. I have also been known to have reviewers and editors prevent me from presenting my views fully in the peer reviewed literature, this does not always mean they have convinced me I am wrong! But anyway, it was hardly a confident prediction in the first place.

2. At the time, more established seasonal forecasters were not predicting El Nino. From Roger's previous post, attributed to NOAA/NCEP/CPC:
Most of the statistical and coupled model forecasts indicate ENSO-neutral conditions in the tropical Pacific through the end of 2006 (Fig. 6). The spread of the most recent statistical and coupled model forecasts (weak La Niña to weak El Niño) indicates uncertainty in the outlooks for the last half of the year. However, current conditions (stronger-than-average easterly winds over the central equatorial Pacific and below-average upper-ocean heat content) support those forecasts indicating that La Nina conditions will continue for the next 1-3 months.

3. An El Nino actually occurred, but it was a weak one.

So, what can we conclude from this?

Well, Hansen clearly went out a little on a limb, at a time when established forecasters were more circumspect. He got it right, with events turning out absolutely in line with his prediction. I'm not sure how Roger justifies his "something for everyone" which is clearly intended as a jibe at Hansen for over-egging things, as the prediction of a super El Nino was only ever presented as a reasonable possibility, not a high probability event. As I've said before, I do believe that scientists bear responsibility when their words and research are predictably misunderstood as a result of them presenting their results in a misleading manner. I don't think this is one of those occasions. His words were admirably clear (admittedly not numerically quantitative).

As Roger himself said at the time of the forecast:
If he is proven right with this forecast, contrary to all of the models and statistics, then his credibility will rise far beyond its already stratospheric levels.
(To avoid the appearance of piling onto Roger for the sake of it, I will mention that although I don't claim great expertise in the area, I actually agree with him here, a point which I may blog about in more detail later.)

11 comments:

Roger said...

James:

Something for everyone means:

If the forecast is taken as deterministic:

1. Yes, there was an El Nino
2. No, it wasn't "super"

If the forecast is taken as probabilistic:

Yes, no, maybe all can be invoked.

James Annan said...

Roger,

Thanks for the clarification, but surely you aren't suggesting that any honest competent reader could have misinterpreted the "likely" El Nino and "good chance" of a super El Nino to have been meant as a deterministic forecast that these things would definitely happen?

If Hansen had said a super El Nino was "likely" then you might (no, would) have a case for claiming that he was predicting this as the most likely outcome, but I don't think I am over-parsing his words by applying my simple and straightforward interpretation to the quotation you provided.

I note that Steve Bloom called you out on this exaggeration three years ago when you first blogged about it.

Equally, Glantz seems to be building up a bit of a straw man in the article you cited. Maybe the press coverage didn't present things fairly, or maybe there's a bit of turf protection going on...

Roger said...

James-

A deterministic forecast is a prediction of the occurrence of an event. It does not mean a prediction of occurrence with 100% probability (that of course is a probabilistic forecast). I'm sure you know this.

A probabilistic forecast can be evaluated using deterministic methods, and vice versa. But you probably know this as well.

Hansen didn't provide numbers, so I don't know what he meant by "good chance of" or "is charged for" a Super El Nino.

Hence, recognizing these uncertainties in exactly what was being forecast, I said in the comments on evaluation that there is something for everyone. On _could_ choose to evaluate the forecast in deterministic or probabilistic fashion.

Hope this clarifies.

James Annan said...

Roger,

In this context, "a good chance" is obviously lower probability than "likely". For starters, the former (super El Nino) is conditional on the latter (any El Nino), and thus logically cannot have higher probability, and secondly, he could easily have said the super El Nino was "likely" if he'd though that appropriate. Thus I see no basis for your interpretation that Hansen thought a super El Nino was more likely to happen than not - maybe there is something else in the draft that supports your interpretation? Hansen was clearly warming of the possibility of an event which while not predicted to occur with high probability, would be high impact if it came to pass.

Roger said...

James-

I don't see the relevance of your parsing of these terms. Again, I don't think it matters.

If I predict that the Pittsburgh Steelers will win the Super Bowl next year based on history (6 of 43, or so), I can do so in deterministic fashion without having a high probability of likelihood in my judgment.

After next season you can evaluate my forecast in different ways.

James Annan said...

The relevance is that under any reasonable interpretation, Hansen thought that a super El Nino was less than 50% likely to happen. This makes your sniping at his supposedly failed prediction of a super El Nino completely bogus.

It is disappointing, but sadly no longer surprising, that you don't think such things matter. They do matter to people who actually do quantitative evaluation of predictions for a living.

Roger said...

James-

Do your really think that forecasts with less than a 50% expressed probability of occurring cannot be evaluated with deterministic methods?

Where did I say Hansen's forecast failed? I gave you a whole range of options for evaluating it, including success.

I have tried to provide your substantive answers to your comments. If you want to see bad faith or some sort of malice in them, I suppose you will. But there are other options.

Chuck said...

How was 2006 an el nino year? The SOI wasn't even -2, and it was significantly higher than any of the previous four years.

P. Lewis said...

The ONI values clearly indicate that a weak El Niño occurred from middle 2006 to early 2007:

2006
JAS 0.5
ASO 0.6
SON 0.9
OND 1.1
NDJ 1.1

2007
DJF 0.8

Hank Roberts said...

An email draft is a forecast.

A "100 percent" expectation can't be determined to have been true or false.

A "good chance" expectation can be determined to have been true or false.

A finished text does not supplant a draft.

I think I'm getting the hang of this.

Hank Roberts said...

But I need help picking one of these to back the interpretation:
http://www.google.com/search?q=define%3Adeterministic