Tuesday, June 19, 2007

The "$20,000" Climate forecast bet

This looks interesting:
Scott Armstrong will announce a $20,000 challenge (each side to post $10,000 to go to a charitable cause nominated by the winner) that he will be able to more accurately forecast climate change than can any fully disclosed climate model.
I found this site via this post on Andrew Gelman's (and colleagues) interesting stats blog (the article he is talking about in that post is worth a read - in fact it bears directly on the frequentist v Bayesian debate). Actually Gelman recently commented on the climate sensitivity issue, albeit in a noncommital way. I look forward to seing if he will deal with it in more detail. But that's an aside.

On further digging, it seems like Armstrong is a common-or-garden variety delusionist, with his "audit of the IPCC forecasts of global average temperature" being little more than a laundry list of wrongheadedness, nonsequiturs and nitpicking, much of which isn't even remotely relevant ("For example, policy responses to Rachel Carlson’s Silent Spring, failed to anticipate that millions of people would die from malaria because efforts to reduce the use of DDT"). In fact he takes a particularly clueless tack with his summary:
They found the IPCC forecasts have no validity and conclude that the there is no more reason to expect global warming over the next 90 years than there is to expect global cooling
since there is no longer anyone remotely credible in the field of climate science who denies that CO2 will have a significant warming effect (even Lindzen has crept up past 1C for a doubling, which any reasonable analysis would convert to at least a probable warming).

There are several obvious possibilities here: either his "$20,000 bet" (only $10,000 from each side, it's no bigger than my existing bet) is just a publicity stunt to get people to visit his website - remember the stupid bet on oil futures, which bought widespread press coverage for someone with a book to sell - or else he is intending to do something supposedly clever and abstruse with time series analysis and end up with a warming of 0.2C/decade for the foreseeable future, just like all the models do. Of course by intelligently analysing a range of model outputs it should be possible to generate a forecast which will probably outperform a model - simply taking their average (strictly, median) guarantees to beat half the models, and can reasonably expect to beat a clear majority.

In principle I'm all in favour of expanding interest in climate forecasting. In fact, in the next few months or years we expect to generate some explicit forecasts of our own (it's what the new apprentice is for). Maybe I'll take on Armstrong if the terms of his bet are reasonable. I'll try to remember to chase it up next week when he officially announces his challenge.


Tim Lambert said...

I think this bit from Armstrong's paper says it all:

"History is filled with the poor treatment of those who attempt to introduce science into arenas where emotions are high and vested interests are threatened. Galileo springs to mind."

James Annan said...

Yes, it's full of...quotable quotes :-) I initially gave him the benefit of the doubt due to Gelman's link (and to be fair there is nothing obviously kooky about the statistical significance thing).

C W Magee said...

Are there any errors associated with the bet?

William M. Connolley said...

Who would have thought that there are 140 principles to forecasting? And that IPCC managed to break 72 of them. I suspect he wrote his own wiki page, too... Why is this anything other than a pile of b*ll*cks?

Note, BTW, the terms of the bet: "he will be able to more accurately forecast climate change than can any fully disclosed climate model". First off, that rules out any averages. Secondly, it rules out, e.g., HadCM3 which isn't disclosed. But why should he insist on disclosure of the model rather than the forecast?

William M. Connolley said...

Sorry, found another wonderful quote rather approriate to forecasting; "Carter, et al. (2006) examined the Stern Review (Stern 2007)."

James Annan said...

That quote's a goodun :-)

I was meaning that the model mean would be a sensible choice for him, to probably beat any one model output. I agree there are lots of potential gotchas in what he has written. I have emailed him for more details...

Marion Delgado said...

lord this is demented - what difference does it make whether the model is fully disclosed? This is not like Deep Blue vs. Kasparov.

has he backtracked and postdicted better than the IPCC's announced consensus (it's not like they have one Grand Unifying Model that all bow to)?

Or is he channeling the wisdom of the market? Anyway, it'd be fascinating to find out what he would say would be a not-fully-disclosed factor that would let some greenie climate alarmist out-predict his "model".