Tuesday, October 18, 2005

Shadowing and chaos

Following on from previous posts on chaos, I'm now going to look at a bit of theory in more detail, in particular the shadowing lemma.

The issue the shadowing lemma addresses is this: given a set of differential equations which describe a chaotic system (such as the Lorenz equations), we have no way of calculating a true trajectory, since all of our numerical methods make various approximations (such as using the finite difference (x(t+Dt)-x(t))/Dt in place of the true derivative dx/dt, for example). Moreover, digital computers only calculate and store results to finite precision, so there are rounding errors at every step. In a chaotic system, these errors will grow exponentially and so the model's trajectory (when initialised from a particular state) will differ wildly from the exact system. So what can we hope to learn from the model?

The shadowing lemma provides a very encouraging answer to this problem. It assures us that, although the true system does not track the model's output when they are initialised from the same starting point, there is a trajectory of the true system (starting from a slightly perturbed initial state) that stays close to the model for an arbitrary length of time. So the model output does in fact "look like" a trajectory of the system after all. The pdf file linked from here is one of the most accessible descriptions I've found on the web (the "hyperbolic system" it refers to is a technical term which includes the standard chaotic systems of classical physics).

Stoat has a nice set of graphs showing the growth of small perturbations in the HADAM3 model (atmosphere component of the HADCM3 atmosphere-ocean GCM). As I mentioned in the comments to his post, I am a little suspicious that a small local perturbation can kick off differences across the whole globe within a day or so. Note that the "model physics" does not support pressure (sound) waves so information should only propagate at around the speed of the flow. It seems likely that the propagation speed in these experiments is instead a numerically-determined rate of one grid box per time step. Fortunately, the shadowing lemma comes to the rescue here. Although the perturbation he used would probably not grow in this way given a numerically precise solution to the fundamental equations, the shadowing lemma tells us that there is a true trajectory of the exact system which looks similar to each model run, and therefore their two sets of initial conditions form a (control,perturbed) pair whose difference really does grow as the plots show. Their initial difference would necessarily be small in magnitude, but I expect it would be globally dispersed in nature.

Now, Professor Eykholt made repeated reference to the shadowing lemma in his emails to me, which you can read on Roger Pielke's blog. (I'm amused to note that they're both happy to publish my email without bothering to ask, which rather puts Eykholt's "totally unethical" accusation into context). I struggled to find a way of interpreting his first comments so as to be somewhat relevant, and I can see now how I misunderstood them as a result. However, given that the shadowing lemma only applies to chaotic systems in the first place, it seems bizarre to attempt to use it to demonstrate that a system is not chaotic. In fact, on re-reading his emails his line of argument appears very strange indeed. It is precisely the shadowing lemma that tells us that there are initially-close trajectories of the real system which diverge in the way that the numerical trajectories do, as per the discussion of HADAM3 above. I cannot see how his statement "The shadowing lemma gives you a scale beyond which small perturbations cease to have any important effects" can be reconciled with what is generally understood about chaotic systems. Unfortunately, he refuses to communicate any further on the matter, so I'll never get to the bottom of what he is thinking.

13 comments:

Anonymous said...

If the point remains within the Basin of Atraction (See: Eric W. Weisstein. "Basin of Attraction." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/BasinofAttraction.html ) then the distance it is moved will not be important.

Cheers, Alastair.

CapitalistImperialistPig said...

James,

OK, I had to get away from Roger's blog before I started pounding my head against the wall, but I still have some questions.

Belette's perturbtion is small in amplitude but large in areal extent. If the models don't have sound, then his perturbation ought to propagate like a long wavelenth (compared to depth of atmosphere) gravity wave, right? Such a wave would have different propagation characteristics from a very short wave gravity wave generated by a butterfly wing flap. It ought to maintain its coherence a lot longer, for one thing. Is it possible that the butterfly wing flaps are much more likely to lie in the stabe manifold because of this?

I have a more technical critique of your shadowing lemma reasoning i'd like to try out later, when I have slightly more time.

Thanks for the link to the Michael Cross notes, by the way. They seem very good.

James Annan said...

Alastair,

The point will return to the atractor but in the case of a strange attractor (click the "attractor" link in that page you cite) it will be at a different position, thus the future evolution will rapidly diverge from the unperturbed case.

CIP,

I particularly like RP's latest demolition of the conservation of momentum (here). But back to your question. In a GCM, a perturbation at the smallest resolvable scales will always diverge. I do believe that the GCM exaggerates the effect of a point source somewhat, but nevertheless, any random perturbation will have a non-zero projection onto the leading lyapunov vector and this component will grow even as the bulk of the perturbation dissipates. Besides, things like energy and momentum are conserved on all scales, so a small disturbance affects the volume average on any grid scale.

CapitalistImperialistPig said...

Re: RP. Ouch! I guess I give. On the plus side, I notice CSU is advertising a tenure track position in Climate Dynamics. Maybe you know someone who should apply. Excellent skiing is fairly close.

I will ignore the fact that I thought you were a bit cavalier in dismissing the possibility that butterfly perturbations could inhabit the stable manifold.

About the shadowing lemma: If A is the original trajectory, A* the numerical trajectory corresponding, and A' the shadowing trajectory of A*; B the "physically perturbed" trajectory, B* its numerical image, and B' a shadow of B*, we know that there is a B' close to B*, and an A' close to A*, but I don't see that you proven that the correlations of A* and B* are similar to the correlations of A and B. Am I right?

James Annan said...

CIP,

You are right but by increasing resolution, we can in principle limit the divergence of numerical model from the underlying continuum equations over a time interval of our choice, so although the "real", shadowing and model trajectories are all distinct, their differences over a finite interval can in theory be bounded to whatever level we choose.

Sure, a random small-scale perturbation will project primarily onto decaying modes. There will only be a handful of growing modes in a 10^7-dimensional model. But (certainly in the theoretical, continuous case) the all-important projection onto the fastest growing mode will not be precisely zero.

Hm...climate dynamics in Colorado could be fun. But would I have to study the new non-conservative "atmospheric physics" or could I stick to the fuddy-duddy version, I wonder...:-)

CapitalistImperialistPig said...

I guess what I'm really curious about is how much we know about the stable and unstable manifolds.

Last time I heard, Tom von der Haar was pretty much the god of CSU meteorology - just in case anybody you know *is* thinking of applying - though my info might be dated.

Anonymous said...

The good news of the shadow lemma may be that there is a trajectory from a slightly perturbed initial state that stays close to your model, but isn't there bad news as well? Doesn't the set of "slightly perturbed initial states" in general diverge so that soon it will cover all of the available phase space? I.e. I can pick any physically possible state of the weather some months later and it is likely to be close to the evolution of one of those initial states. In that case I don't think the shadow lemma gives you much help.

James Annan said...

CIP,

It's known that the growing space (ie the manifold where reality "lives") is low-dimensional - eg D. Patil, B. Hunt, E. Kalnay, J. Yorke, E. Ott, Local low dimensionality of atmospheric dynamics, Phys. Rev. Lett. 86 (2001), 5878-5881. But note that they are only interested in "meteorologically interesting" mesoscale perturbations, and stoat's pics illustrate that there is also a much more local convective mode (that weather forecasters are not generally interested in). See also J. D. Annan. On the orthogonality of Bred Vectors. Monthly Weather Review, Vol 132, No 3, pp 843-849 2004 for an eggregiously irrelevant self-citation in this field :-)

Thomas,

The SL doesn't "help" in the sense of directly telling us where to find reality or how to make a better forecast. It helps in the context of this argument, by assuring us that the model is behaving in a qualitatively realistic way.

Anonymous said...

James,

If a point is in the basin of attraction of a strange attractor, its path will be that strange attractor. Perturb the point using a butterfly's wings and it will still enter the same strange attractor althoough its path will be completely (no two points will coincide) from the first path. This lack of coincidence is not possible using a computer which has a finite precision.

If the point is placed on the cusp of two strange attractors, then the flap of a butterfly's wings could determine which attractor is entered, and hence the flap of a butterfly's wings in Brazil COULD cause the system to enter a strange attractor in which a tornado happened in Texas. However, a hurricane in the Carribean is more likely.

Cheers, Alastair.

James Annan said...

Alastair,

There is some vague approximation to the truth in some of what you say, but you are wrong to talk of 2 strange attractors - better to think of it as one, with many different lobes (like Lorenz but more complex). The small pertubation means that the future states of the (control, perturbed) pair will be in unrelated lobes, with no correlation between the two states.

CapitalistImperialistPig said...

James,

The Patil, Hunt, Kalnay, Yorke, and Ott paper seems to be online at Eugenia's U MD site here

CapitalistImperialistPig said...

And James,

How can you be sure that the butterfly wing flap is in a locally low dimensional BV space (to use their terminology)?

James Annan said...

CIP,

That seems like much the same question again in different terminology. The flap will not lie entirely (or even largely) in this space. but so long as it has a non-zero projection onto it (which it will, unless specifically designed not to), that part will grow. One thing that wmc's experiment illustrates is that there is a highly local and very rapidly growing mode (convection), which a point perturbation is well-equipped to set off. This is generally considered a (controllable) problem for weather forecasting, but it's a positive benefit from the POV of this discussion.