New paper

Those who keep a close eye on my work page (yeah, right) will have noticed a new preprint appeared there recently: Efficiently constraining climate sensitivity with paleoclimate simulations. SOLA Vol 1 pages 181-184 2005. It's only a short 4-page summary in a rapid response journal (just right for sneaking inside the IPCC deadlines) but breaks new ground in 3 ways which I hope will help to point the way forward for climate prediction.

• We use an efficient multivariate parameter estimation method with a state-of-the-art GCM for the first time. This is many many orders of magnitude more efficient than multifactorial sampling, and several people have told us over the last couple of years that it couldn't be done. (I won't deny it has some potential drawbacks, though - but we haven't found any show-stoppers).
• We have tried to take some account of model inadequacy - by which I mean the fact that there are no "correct" parameters for which the model actually looks just like the real world, so the standard "perfect model assumption" is wrong.
• We have used out-of-sample data (in this example, a simulation of the Last Glacial Maximum) to attempt to improve the rigour with which predictive skill is assessed. Merely managing to fit a set of data doesn't automatically mean that the model will skillfully predict climate under a different forcing!
  

All of these 3 aspects are approached in a rather simplistic manner, and all can (and no doubt will) be improved, eventually by us if no-one else does it first. But it's good to have a benchmark, if only a rather low one, to shoot for in subsequent work.

Oh, by the way, we didn't really learn anything startling about climate sensitivity. But we all know that's about 3C anyway :-)

The Blue Planet Prize

Every year, the Asahi Glass Foundation awards "The Blue Planet Prize" to two individuals or organizations "who have made major contributions to solving global environmental problems". The winners are invited to Tokyo where they give public lectures on their work, and we usually go along to listen (and perhaps do a bit of shopping in the Big Smoke).

One winner is usually a top-notch scientist in the traditional mold (past winners include Wally Broeker, Suki Manabe, Robert May) and the other more of an activist or politician (Vo Quy, Gro Harlem Brundtland, various worthy organisations I may have vaguely heard of). This year, Nick Shackleton and Gordon Hisashi Sato were the joint winners. They talked at some length about their work, which was very interesting, but then both ended with rather incongruous calls to cut CO₂ emissions. The sentiment may be well-meaning but it all seemed a bit odd.

However, their comments were nowhere near as odd as the introductory film we were shown for the first time this year, which consisted of pictures of war, devastation, pestilence and natural beauty all overlaid with an inane voice-over asking cheesy things like "Why are we killing Mother Earth". It would have been rude to laugh out loud, but I couldn't suppress the odd snigger.

I hope they received enough negative feedback that next time they'll revert to just having the talks!

The Guardian on prediction markets

A short article in the Grauniad a couple of days ago concerning prediction markets:

Markets have now been shown to out-perform collections of "experts" in predicting monthly economic data better than the best economists, or in accurately forecasting whether WMD would be found in Iraq. Mathematicians believe that the existence of a market allows large numbers of people effectively to "pool" their collective wisdom, with each new trader able to bring a unique new insight into the mix.

How can this power be harnessed for the social good? Iowa has recently been experimenting with the trading of "flu predictions", in which traders have been asked to forecast the state of flu alert that will be in place in the state several weeks in advance. Researchers have found that the flu prediction market has been able to predict the state of flu alert with 90% accuracy a fortnight ahead of time - long enough to allow medical staff to gear up prevention and treatment regimes. Iowa economists will next turn their attention to a market that will predict the number of bird flu cases in the US. It cannot come soon enough.

Shadowing and chaos

Following on from previous posts on chaos, I'm now going to look at a bit of theory in more detail, in particular the shadowing lemma.

The issue the shadowing lemma addresses is this: given a set of differential equations which describe a chaotic system (such as the Lorenz equations), we have no way of calculating a true trajectory, since all of our numerical methods make various approximations (such as using the finite difference (x(t+Dt)-x(t))/Dt in place of the true derivative dx/dt, for example). Moreover, digital computers only calculate and store results to finite precision, so there are rounding errors at every step. In a chaotic system, these errors will grow exponentially and so the model's trajectory (when initialised from a particular state) will differ wildly from the exact system. So what can we hope to learn from the model?

The shadowing lemma provides a very encouraging answer to this problem. It assures us that, although the true system does not track the model's output when they are initialised from the same starting point, there is a trajectory of the true system (starting from a slightly perturbed initial state) that stays close to the model for an arbitrary length of time. So the model output does in fact "look like" a trajectory of the system after all. The pdf file linked from here is one of the most accessible descriptions I've found on the web (the "hyperbolic system" it refers to is a technical term which includes the standard chaotic systems of classical physics).

Stoat has a nice set of graphs showing the growth of small perturbations in the HADAM3 model (atmosphere component of the HADCM3 atmosphere-ocean GCM). As I mentioned in the comments to his post, I am a little suspicious that a small local perturbation can kick off differences across the whole globe within a day or so. Note that the "model physics" does not support pressure (sound) waves so information should only propagate at around the speed of the flow. It seems likely that the propagation speed in these experiments is instead a numerically-determined rate of one grid box per time step. Fortunately, the shadowing lemma comes to the rescue here. Although the perturbation he used would probably not grow in this way given a numerically precise solution to the fundamental equations, the shadowing lemma tells us that there is a true trajectory of the exact system which looks similar to each model run, and therefore their two sets of initial conditions form a (control,perturbed) pair whose difference really does grow as the plots show. Their initial difference would necessarily be small in magnitude, but I expect it would be globally dispersed in nature.

Now, Professor Eykholt made repeated reference to the shadowing lemma in his emails to me, which you can read on Roger Pielke's blog. (I'm amused to note that they're both happy to publish my email without bothering to ask, which rather puts Eykholt's "totally unethical" accusation into context). I struggled to find a way of interpreting his first comments so as to be somewhat relevant, and I can see now how I misunderstood them as a result. However, given that the shadowing lemma only applies to chaotic systems in the first place, it seems bizarre to attempt to use it to demonstrate that a system is not chaotic. In fact, on re-reading his emails his line of argument appears very strange indeed. It is precisely the shadowing lemma that tells us that there are initially-close trajectories of the real system which diverge in the way that the numerical trajectories do, as per the discussion of HADAM3 above. I cannot see how his statement "The shadowing lemma gives you a scale beyond which small perturbations cease to have any important effects" can be reconciled with what is generally understood about chaotic systems. Unfortunately, he refuses to communicate any further on the matter, so I'll never get to the bottom of what he is thinking.

Still flapping

As an update to my previous post, Prof Eykholt has emailed to say that I have misrepresented him. However, although of course I apologise for any embarassment caused, it is not actually clear to me how I actually misinterpreted his email to me. Some of his original comments seemed a bit unclear to me, but I thought I had managed to interpret them in a way which was consistent with what I understood of chaos theory (I did email him back to check, but he didn't reply to that). I find it hard to believe that he really means to support Roger Pielke's position that small pertubations will vanish completely in a chaotic system. After all, Prof Eykholt is also quoted as saying (on Roger's blog):

The butterfly effect refers to the exponential growth of any small perturbation.

So I am now more puzzled than ever as to what he really means. I've asked him and Roger if they can supply any theoretical or practical support for Roger's claim that below a certain size, small pertubations fail to affect chaotic systems (at least, the atmosphere in particular)...

Update

Ok, on re-reading Prof Eykholt's emails, I can now see how I misinterpreted them, and I'm sorry for any confusion I have caused. In my defence, I did email him back to check that I had understood his point correctly (he explicitly invited me to do so) last Thursday, and never received a reply. I still think that my comments on chaos are entirely uncontroversial and in line with the overwhelming maority of the field. Prof Eykholt's emails to me lean heavily on the Shadowing Lemma, so I'll probably do a post about this shortly, explaining its relevance (or otherwise) to the debate.

More on the butterfly flap

(See here for previous.) Well, RP Snr's promised followup post consisted of an odd quotation from Richard Eykholt who works in the physics department of Colorado State University. His quotation contains the comment that

"The point of the [exponential growth of minuscule perturbations] effect is that it prevents us from making very detailed predictions at very small scales, but it does not have a significant effect at larger scales."

which RPSnr takes as vindication of his claim that small perturbations do not affect the real world, only models.

The quotation seemed a bit unclear, and RPSnr's position is clearly wrong, so I emailed Prof Eykholt to see what he really meant. He explained that he was referring to issues of practical prediction, in which case any additional perturbation of a similar size to the rounding error of a finite precision computer model cannot harm predictability over broad spatial scales, since the model already has lots of perturbations of equal size (not to mention more substantial errors) which limit predictability. Of course this is true (to the point of triviality), but it hardly seems relevant to RPSnr's claims. Further, Prof Eykholt explicitly agreed that a small random perturbation in a chaotic system will grow to a point at which the original and unperturbed systems are completely decorrelated.

Meanwhile, back in the comments section of his post, RPSnr seems so determined to defend his untenable position that he is now claiming that not only turbulent energy but even momentum itself will dissipate in the real world...

CAN YOU SPOT THE REAL SCIENTIST?

This comment on Slashdot is worth a read (thanks Michael Tobis).

The "Butterfly effect"

There's recently been a bit of a flap on some blogs concerning the "butterfly effect". As that wikipedia page indicates, this is the poetically-named phenomenon whereby chaotic systems (such as the atmosphere) exhibit sensitive dependence on initial conditions. That is, if you had two identical copies of the same system, and perturb one ever so slightly, the perturbation will grow over time and eventually the states of the two systems will be completely uncorrelated. Its discovery and significance in the context of atmospheric science was first noted by Lorenz in his famous 1963 paper:This picture (snaffled from this page) shows some results from Lorenz's original paper.

Less widely-known than Lorenz's work is the fact that the chaotic dynamics of the 3-body problem had been shown much earlier by Poincare. Following the links on wikipedia (and googling) should cover the historical and technical background adequately.

The significance of chaos to weather prediction is sometimes overstated - the existence of a Lyapunov (watch the spelling, wmc!) exponent of > 1 does not in itself preclude accurate long-term predictions, since it relates only to infinitesimal perturbations integrated over infinite time (note that we are staying in the realm of mathematics and classical Newtonian physics here - I'm not going to start talking about molecular and quantized effects). In realistic applications, it is the current (and near-future) growth rate of finite perturbations that is more important, and this may well be negligible or negative over significant periods of time. And even that ignores the effect of model error, which means that even a perfectly initialised model state will inevitably diverge from truth over coming days. Some analysis of weather prediction I've seen indicates that model error is a substantial source of forecast error, with chaos relatively unimportant. But this is not my current field of interest, and this is also probably getting a bit obscure...

Anyway, the point is that a random perturbation will have a non-zero projection onto Lyapunov vectors with exponents greater than 1 (ie, growing vectors). In the atmosphere and many other chaotic systems, most of the perturbation will project onto decaying vectors, so overall it will initially shrink. However, ultimately the growing component will come to dominate and the pertubation will grow (with probability 1).

This is the sort of thing that is easy to show with numerical models of the atmosphere. Simply perturb a reference run, and see what happens. There are a couple of "gotchas" that could confuse the results. Firstly, limited numerical precision means that a small pertubation could fall below the level of numerical precision and be rounded out of the system. This is demonstrated by running the model at diffferent precisions. A perhaps more subtle problem is that many codes contain max and min operators, to prevent nonphysical solutions (eg turbulent energy is not allowed to fall below a threshold, velocities may be limited for computational reasons). If a variable crosses such a threshold and is reset to the limit, then a slightly perturbed version will be reset to the same point and again the perturbation is lost. But these are nit-picks in a result that is otherwise robust across all adequately realistic models. It is possible in theory to devise a perturbation that shrinks even without relying on these factors (align it precisely with only the shrinking LV), but a random pertubation will not have this property. Of course the existence of an unknown butterfly flapping its wings has no direct bearing on weather forecasts, since it will take far too long for such a small pertubation to grow to a significant size - if it affects the weather 100 years from now, this will be obvious in the atmospheric state 99 years and 51 weeks hence, and so will be accounted for in the forecast.

I might as well point out again here that the chaotic sensitivity of a system has no direct bearing on forecasting the climate (ie the long-term statistics) of the system. To copy from my comment on realclimate:

Not only is the climate of the Lorenz model easy to understand, it is also simple to predict how it will respond to a variety of "external forcings", in the form of either a parameter perturbation or direct forcing term in the dynamical equations. Eg see here and here. However the detailed trajectory is unpredictable except in the very short term.

Anyway, this is all pretty well-known stuff - or so I thought, before reading Roger Pielke Sr's blog. He seems to deny that this "butterfly effect" exists at all. Myself, Gavin and William have all pulled him up on it, so it will be interesting to see how he justifies this in his promised follow-up post.

BBC NEWS | Science/Nature | Europe ice mission lost in ocean

You have to feel sorry for the scientists who saw several years of work go bang in a few minutes. The Guardian describes it as a disaster, which seems a bit strong given that there were 18,000 deaths following the earthquake on the same day. But it's certainly a shame.