So our comment, Nicola Scafetta's comment, Reto Knutti's comment, and Steve Schwartz' reply (combined to all of us) are all on line and some people seem to be getting very excited by it all. In his reply, Schwartz was quick to jump at Scafetta's suggestion that the "pertinent time constant" can actually be diagnosed as about 8y, or maybe 12y, and seems happy to admit that his original analysis (5y) was wrong. Unfortunately, the reviewer(s?) and Editor gave him free rein to present a completely new analysis, based on a new model - hardly the point of a Reply, I thought - which is pretty much just as bogus as the original although the numbers don't turn out quite as absurd. The basic point, that we made in our comment, is that it is trivial to check that such proposed "novel" analyses actually diagnose something useful for systems where the answer is known in advance (and furthermore, whether such extreme simplifications have some chance of capturing any useful information about complex systems), and such testing should normally be something that a researcher should have a go at themselves, before claiming to have overthrown a few decades of climate science. It will be no surprise to anyone who actually works in the area that the new analysis fails just as dismally as the last one, and for much the same reasons.
I can't be bothered with a detailed analysis so will just highlight what appears to be the most immediately fatal flaw in the whole idea. The analysis rests on the claim that the climate system can in fact be characterised by two well-separated time constants, and further that these can be diagnosed from the time series of global mean surface temperature. The foundation for this second claim seems to based on a test case where Scafetta attempts to fit his curve to the output of a synthetic time series with known properties. He used a time scale of 12y for the test, and gets an estimate of about 8y out (which matches his analysis of the real climate data), and thus claims that the real answer is either 8 ± 2, or 12 ± 3 - and apparently sees no irony in the fact that these two answers don't even overlap. There is no indication of what these ± values are supposed to indicate, and it is apparent to the naked eye that his lines (in all of his figures) are not actually best fits to the data. Despite this evidence of bias, he finally presents the the lower value of 8 as the "observed value" in his conclusions, only mentioning 12 as a "hypothetical" possibility even though his own analysis, limited as it is, indicates that higher value as a best estimate (after correcting for his estimate of the bias) and admits some additional uncertainty above that value.
So his test is kind of like what we did in our comment, except we did it properly, using a wide range of time scales (5-30y) in the synthetic time series and looking not only at the mean bias but also the uncertainty of a single replicate, in order to get a good handle on how the analysis performs. Of course, we found that both the bias and the uncertainty grow for the larger time scales, which follows directly from known properties of the (Bartlett's rule) method. In fact at the highest-valued time scale of 30y (corresponding to a sensitivity of 6C in this simple model) the analysis generates an estimate that is typically less than half the true value and quite possibly as low as 5y. The obvious conclusion is that you can't reliably diagnose the time constant of an AR(1) time series by this method unless the length of data set available is much much longer (by a surprising margin) than the characteristic time scale of the system. Adding in another two free parameters, as Scafetta does, can hardly improve matters, and neither does using the monthly data (even though Schwartz says this adds "many more independent data points", it clearly does nothing of the sort given a decorrelation time scale of several years).
Lucia in the posts linked above has taken it upon herself to check the accuracy of the method. Her latest post indicates at least that she is starting to look on the right lines, although she's not quite got there yet and her belief that the uncertainty "should drop dramatically" when she uses monthly data is mildly amusing. Eventually, if and when she checks the method against synthetic data with a really long time constant, she will probably realise that the output of the analysis is so inaccurate that it doesn't tell us much at all. And remember that this is in the best possible case where the data actually are generated by a system which exactly satisfies the hypothesis of a simple autoregressive system with white noise forcing. Once you consider that the real world is rather more complicated (both in terms of multiple time scales of response, and the strong but non-linear external forcing) it is a bit of a lost cause.
So, if Lucia does her homework properly, she ought to get there in the end. Whether or not she will have the manners to retract her stupid statement that "two of the criticisms are flat out wrong" (in our comment) remains to be seen.
Steve Schwartz also looked up a method (from Quenouille, 1949) that reduces the bias of Bartlett's Rule in estimating the autocorrelation coefficients. What he failed to observe in his reply is that although the mean bias is indeed reduced by this approach, the uncertainty gets larger, which (depending on the specifics) roughly compensates. That is, one can still get estimated time constants that are much smaller than the true value, but the modified method can also generate estimates that are much larger, with the estimated correlation often exceeding 1. But instead of any meaningful analysis about how this impacts his results, Schwartz prefers instead to waffle on about Einstein and electrical circuits, presenting yet another simple model (different from Scafetta's) without making any attempt to explore how well it can either be identified from the data (it can't) or matches any plausible model of the climate system (it doesn't). I would have hoped that any competent scientist could have worked that out for themselves prior to even bothering to submit this sort of stuff for publication, but climate science work in mysterious ways sometimes.
It is curious how sceptics are quick to dismiss numerical models that actually represent the broad details of the atmospheric and oceanic circulations reasonably well, in favour of some simple approximations that make no attempt to do so. The issue here of course is not whether the models are "correct", but whether a method that makes no detailed assumptions about the behaviour of the climate system (only really requiring that it conserves energy) can actually diagnose the behaviour of any system, simple or complex. Schwartz' and Scafetta's various methods fail dismally on all counts.
I can't be bothered with a detailed analysis so will just highlight what appears to be the most immediately fatal flaw in the whole idea. The analysis rests on the claim that the climate system can in fact be characterised by two well-separated time constants, and further that these can be diagnosed from the time series of global mean surface temperature. The foundation for this second claim seems to based on a test case where Scafetta attempts to fit his curve to the output of a synthetic time series with known properties. He used a time scale of 12y for the test, and gets an estimate of about 8y out (which matches his analysis of the real climate data), and thus claims that the real answer is either 8 ± 2, or 12 ± 3 - and apparently sees no irony in the fact that these two answers don't even overlap. There is no indication of what these ± values are supposed to indicate, and it is apparent to the naked eye that his lines (in all of his figures) are not actually best fits to the data. Despite this evidence of bias, he finally presents the the lower value of 8 as the "observed value" in his conclusions, only mentioning 12 as a "hypothetical" possibility even though his own analysis, limited as it is, indicates that higher value as a best estimate (after correcting for his estimate of the bias) and admits some additional uncertainty above that value.
So his test is kind of like what we did in our comment, except we did it properly, using a wide range of time scales (5-30y) in the synthetic time series and looking not only at the mean bias but also the uncertainty of a single replicate, in order to get a good handle on how the analysis performs. Of course, we found that both the bias and the uncertainty grow for the larger time scales, which follows directly from known properties of the (Bartlett's rule) method. In fact at the highest-valued time scale of 30y (corresponding to a sensitivity of 6C in this simple model) the analysis generates an estimate that is typically less than half the true value and quite possibly as low as 5y. The obvious conclusion is that you can't reliably diagnose the time constant of an AR(1) time series by this method unless the length of data set available is much much longer (by a surprising margin) than the characteristic time scale of the system. Adding in another two free parameters, as Scafetta does, can hardly improve matters, and neither does using the monthly data (even though Schwartz says this adds "many more independent data points", it clearly does nothing of the sort given a decorrelation time scale of several years).
Lucia in the posts linked above has taken it upon herself to check the accuracy of the method. Her latest post indicates at least that she is starting to look on the right lines, although she's not quite got there yet and her belief that the uncertainty "should drop dramatically" when she uses monthly data is mildly amusing. Eventually, if and when she checks the method against synthetic data with a really long time constant, she will probably realise that the output of the analysis is so inaccurate that it doesn't tell us much at all. And remember that this is in the best possible case where the data actually are generated by a system which exactly satisfies the hypothesis of a simple autoregressive system with white noise forcing. Once you consider that the real world is rather more complicated (both in terms of multiple time scales of response, and the strong but non-linear external forcing) it is a bit of a lost cause.
So, if Lucia does her homework properly, she ought to get there in the end. Whether or not she will have the manners to retract her stupid statement that "two of the criticisms are flat out wrong" (in our comment) remains to be seen.
Steve Schwartz also looked up a method (from Quenouille, 1949) that reduces the bias of Bartlett's Rule in estimating the autocorrelation coefficients. What he failed to observe in his reply is that although the mean bias is indeed reduced by this approach, the uncertainty gets larger, which (depending on the specifics) roughly compensates. That is, one can still get estimated time constants that are much smaller than the true value, but the modified method can also generate estimates that are much larger, with the estimated correlation often exceeding 1. But instead of any meaningful analysis about how this impacts his results, Schwartz prefers instead to waffle on about Einstein and electrical circuits, presenting yet another simple model (different from Scafetta's) without making any attempt to explore how well it can either be identified from the data (it can't) or matches any plausible model of the climate system (it doesn't). I would have hoped that any competent scientist could have worked that out for themselves prior to even bothering to submit this sort of stuff for publication, but climate science work in mysterious ways sometimes.
It is curious how sceptics are quick to dismiss numerical models that actually represent the broad details of the atmospheric and oceanic circulations reasonably well, in favour of some simple approximations that make no attempt to do so. The issue here of course is not whether the models are "correct", but whether a method that makes no detailed assumptions about the behaviour of the climate system (only really requiring that it conserves energy) can actually diagnose the behaviour of any system, simple or complex. Schwartz' and Scafetta's various methods fail dismally on all counts.
1 comment:
Nothing to add; I just wanted to see at least one comment about "Comments about comments" ....
:-)
Post a Comment