July was a good month for Corbyn's monthly forecast - the rainfall of 167% was in his predicted range of 160-250%, and the mean daily max (anomaly) of -0.8C was also in his range of -1 to -0.2 (data here). The various specifics were largely wrong but I can't go cherry-picking what to evaluate after the event. [I did initially say I would use mean, not max, temp but that was before being pointed to the max data on Philip Eden's broken site.]
So now he is up to 6 correct forecasts out of 12 so far this year (temp and precip for 6 months, June being curiously absent). That still represents a p-value of less than 2% based on his claimed 80% accuracy. That is, if his forecasts really had a long-run probability of 80% of validating, then the probability of getting 6 or fewer correct out of 12 is under 2%. On the other hand, it is better than chance, given that his ranges generally cover less than 50% of a reasonable climatological range. (That doesn't mean much, given that he only publishes the forecast a day or two into the month. The very heavy rain at the start of July was easily predicted by everyone at that point. Also, there is the question as to why June's forecast was not released...but there is little point in speculating on that point.)
So now he is up to 6 correct forecasts out of 12 so far this year (temp and precip for 6 months, June being curiously absent). That still represents a p-value of less than 2% based on his claimed 80% accuracy. That is, if his forecasts really had a long-run probability of 80% of validating, then the probability of getting 6 or fewer correct out of 12 is under 2%. On the other hand, it is better than chance, given that his ranges generally cover less than 50% of a reasonable climatological range. (That doesn't mean much, given that he only publishes the forecast a day or two into the month. The very heavy rain at the start of July was easily predicted by everyone at that point. Also, there is the question as to why June's forecast was not released...but there is little point in speculating on that point.)
5 comments:
Instead of comparing to random chance, shouldn't you compare to the met office or the weather channel or whoever it is that publishes long range forecasts in the UK?
Well, primarily I'm comparing to Corbyn's claimed 80% to see how credible that is. I agree that the "better than chance" comment I made is a bit meaningless. If I had other equivalent quantitative monthly forecasts available for a direct comparison it might be interesting, but I don't know of any. Eg, I don't think UKMO's quantitative monthly outlook is publicly available, though I'm not entirely sure. Certainly the BBC long-range forecast is vague.
James, I'm presuming you are refering to a link in the second comment when you refer to Philip Eden's broken site? Which goes to a Real Climate comment? Real Climate right now apparently has a sitemeter problem if you are using the IE browser.
Chuck, at the moment your site can't be viewed by anyone using IE. (Like me!)
SF, I was trying to link to the first of Adam's two comments (15th and 16th July) where he mentioned the link and that firefox (my browser of choice) couldn't see the link. However I find that the links to specific comments don't work properly (not sure if this is a firefox or blogger problem). Sorry about that.
One thing that's come out of the UKWW comparison (I don't know how often, I've not counted), is that there are times when PC gets the general weather conditions correct, but the synoptic descriptions totally wrong.
The conditions may be all the public are interested in, but it does suggest that his method isn't as good as would initially seem for predicting synoptics.
Anyway as I've pointed the UKWW link here and here there, this is repeating myself.
Post a Comment