Sea Ice forecasts and outcome

I've not really paid that much attention to the sea ice area myself, but I did take a punt on Lab Lemming's betting pool, mainly because (at the time of my bet) there seemed to be a pretty big hole in the bets close to where the sea ice graph seemed to be heading (based on nothing more than eyeballing the graph). So without any detailed calculations, I chose a value in the middle of the hole, to maximise my chances of winning. I didn't quite win, but was one of the closest few.

More interesting than that, is the performance of the "pros". I use the inverted commas because as far as I know, there is no real history of producing annual forecasts - and neither is there any customer - so these people aren't really experienced forecasters, rather they are sea ice modellers and experts of some sort or another. Note that weather forecasting certainly makes heavy use of both numerical models and experts in atmospheric science, but on top of that it involves experts in prediction. Anyway, here is the final sea ice prediction of the experts, issued in August and based on July data:

Clicking on the pic should get a bigger version, or you can go here for more details.

SEARCH have now published the final outcome, including a comparison with the July forecasts. That earlier set of forecasts happens to be the only month that contains a prediction which exceeds 5, which I'm sure is entirely unrelated to their decision to cherry pick that month. Note also that SEARCH are using a different observational analysis which gives a slightly higher observed value (5.36 versus the 5.25 of Lab Lemming). Also they are using monthly mean rather than daily min. None of this, however, obscures the fact that the forecasts are all wrong, and most of them are very very badly wrong.



Looking at the IARC-JAXA graph, it seemed very obvious through July that the sea ice extent was towards the low end of the pack, but well above the 2007 track and not clearly lower than the other years (the ticks mark the start of the month):

So I don't know what basis the researchers had for expecting such low values. SEARCH helpfully adds that many of the forecasts had their own uncertainty estimates of about 0.4, which means that almost all of them failed dismally to even include the observed value in their predicted range, and one might have been as far as 6 sigma out (interpreting the 0.4 value as 2 sigma). Back to the drawing board for them!

Turning briefly to the "semi-pros", Stoat has a bit of a gloat, although he still managed to lose a bet with Bob Grumbine. However, both of them were a lot further off than me...(but to be fair, they made their bets much earlier too).