Wednesday, June 20, 2007

More on the "$20,000" bet

Following on from this post, I emailed J Scott Armstrong, he pointed me to this web site/blog and the bet is outlined in more detail in this post.

At a glance, he is using the most obvious and trivial trick, that he appeared to have ruled out with his talk of forecasting climate change on this page. In fact, the terms of his challenge refer to forecasting annual mean temperatures at a handful of points, using raw model output. The trivial trick here is that of course the models do not directly represent local temperature (typical resolution is ~300km horizontally) and they also have significant regional biases, so meaningfully relating their output to local temperature requires at a minimum some sort of bias correction and/or downscaling. Such bias adjustment is an entirely routine procedure in many branches of forecasting, it is inconceivable that Armstrong does not realise this.

The other big problem is the time scale: the bet is for 1-10 year forecasts. While there are probably some people who can produce usable forecasts over at least the seasonal to annual time scale (and maybe further in some cases), on the whole these aren't the same people as those doing 100 year projections. The GCMs used in the IPCC report don't have any proper initialisation scheme that would enable them to make meaningful annual forecasts, and no-one has ever claimed that they do. From their point of view, whether one year is warmer than the last is basically a matter of chance, and a "persistence" forecast is a pretty reasonable reasonable choice.

A much fairer test of the models would be to look at something like a 20 year trend for global mean temperature (and possibly at a more regional scale: I haven't looked in detail at this). Armstrong claims to be amenable to altering his terms: I've emailed him with these points and will report on his response. Based on what I have read, I'm not optimistic. It reads like a cheap publicity stunt rather than serious challenge.


Unknown said...

Is there anyone willing to bet that one of the years 2007-2010 (inclusive)will not be warmer (by the NCDC summary) than 1998? If so, I think this is a fair bet for both sides, though I'm not sure what it might prove, if anything, except a willingness to put one's cash where one's mouth is.


Brian said...

Armstrong says "(without human adjustments to the model’s forecasts)" so he's probably ruling out downscaling or bias correction.

I'm not sure I understand the time periods - is he saying for the ten-year bet to use measured temps at certain places between 2008-2017 to forecast 2018-2027 results, and compare that to what a model will tell us now? Or to what a model in 2017 will predict? Why not use 1998-2007 data and do the same thing without waiting 10 years.

Fergus, I don't think your bet is fair to the consensus position. 1998 was an anomaly, so it's likely that one or more of the four years you mention will be lower. Multi-year averaging is a much less noisy way to bet.

James Annan said...


I think it's a 10 year ahead, one year mean - eg what will the temp be in 2018, forecast at the end of 2008. Even with a decade of warming, annual variability kills you.

Fergus' bet is not that all these years will be warmer than 1998, but that at least one will be. I've talked about this somewhere previously (here and here) - if none of these years beat the 1998 record then it will start to look like anthropogenic warming is not quite as strong as most believe. OTOH it will hardly refute the whole concept, and it also depends on the data set - some teams think that 2005 was already warmer.

Unknown said...

brian; it's fairly straightforward: if the trend is for warming, then sooner or later, 1998's anomalous warmth must be clearly exceeded (this allows that 2005 might or might not have been as warm or warmer; if it was, it wasn't sufficiently so to eliminate all uncertainty). All I'm suggesting is that this will happen in the next four years.

This is not a scientific gamble, as internal variability of the system makes it very uncertain, but it is simple and understandable. Think of it as betting on a jump race rather than a flat race: in a flat race, the odds will favour the fastest horse, with small variables to consider. In a jump race, the odds are always higher, because there are more uncertainties involved.

If I had $20,000, I'd happily take up someone on this: I reckon at least one of the next four year (including this one) will exceed 1998. But I am poor, so the offer is in principle rather than in cash. For a skeptic with a sense of adventure, this is a decent bet. For a 'consensus' thinker, the odds are worse, but it isn't intrinsically unfair. After all, gambling is no fun if you only ever bet on one-horse races.