I dunno, spend a couple of weeks out of the country and a proper debate strikes up for a change. Marotzke and Forster published this paper basically arguing that the AR5 models don't generally either over- or underestimate temperature changes, based on various trends within the last century or so (up to now). My first thought on a superficial glance at the paper was that it wasn't really that useful an analysis, as we already know that the models provide a decent hindcast of 20th century temps, so it's hardly surprising that looking at shorter trends will show the models agreeing on average over shorter trends too (since the full time series is merely the sum of shorter pieces). That leaves unasked the important question of how much the models have been tuned to reproduce the 20th century trend, and whether the recent divergence is the early signs of a problem or not. (Note that on the question of tuning, this is not even something that all modellers would have to be aware of, so honestly saying "we didn't do that" does not answer the question. But I digress.)
One limitation of the MF study was that they do not know the forcing for each model, or the α and κ parameters, so had to estimate them by linear regression based on (among other things) their temperature time series. Along comes Nic Lewis, and says "aha - this is circular". Now I haven't had time to look into it in detail, but this argument clearly has some validity in principle. MF replied here, but I'm not really impressed by what they said. There is a certain amount of talking past each other - MF are saying that their method is physically reasonable (which it is) and, in previous work, gives good results, however they don't really address the main criticism. Lewis says their method is simply invalid, because the assumptions underlying the statistical theory are violated. In that respect, it is worth pointing out that probably just about all statistical analysis is simply invalid at some level, since the assumptions are rarely precisely correct. Exact linearity, independent errors, gaussian statistics? You've got to be joking. These are never more than approximations to the truth, but hopefully the approximations are good enough that the end result is useful.
Some of the commenters on the climate lab book thread seem to have got a good handle on it. Just to expand on it a bit, if MF got the "correct" values for forcing, α and κ then it wouldn't matter where these numbers came from. However, there will be some uncertainty/inaccuracy in the estimates they have derived, and these did come from the temperature time series, and will lead to some circularity. So the question is really whether these inacuracies are big enough to matter. I have an idea for investigating the magnitude of the problem (which a priori could either be negligible or could indeed invalidate the work). I'm not sure it's really worth the effort though.
17 comments:
I doubt it is worth the effort.
I'd say share the idea with M, F, and/or NL, if you haven't already.
They can decide for themselves how much they want to test the work. For the bigger picture, I doubt it matters much, but, /shrug. This is science, and you can be as careful and precise as it takes in order to satisfy your curiosity and skepticism. Assuming it's satisfiable, that is.
James,
I think you are missing the point. The question is not whether the parametrisation of the models invalidates M&F's work. It is: are the models correct as M&F imply, despite the fit to 20th Century climate being due, as you seem to admit, to parametrisation!
The question is more are the models reasonable given the data we have from the recent past. Reasonable and correct are very different.
Eli loves the /shrug
I got drawn into the debate more than many others. The argumentation was mainly about alleged circularity in the values of forcing extracted from the CMIP5 database. That particular criticism does not have much merit in my view, but otherwise the model may be too crude and badly constrained to tell much.
Eli,
Of course the models are reasonable given the data of the recent past, because as James seems to admit they have been tuned to fit the recent past. The tuning has gone under the name of parametrisation using the most up to date and reliable information, viz that from the recent past.
But the models cant't reproduce the the lifting condensation level, nor abrupt climate change, nor the tropical lapse rate , nor the PETM, nor the Miocene climate, nor the Earth's radiation balance, nor close the surface balance, nor produce a single ITCZ, etc. In fact, if science was not so specialised, it would be in crisis.
What would be more interesting (and amazing) is if the amateur skeptic blowhard blogosphere analyzed a scientific paper on climate and declared it totally correct. Surely there must be plenty of papers they've come across that, with their genius analysis, passed all of their "audits"? It's odd that seemingly every damn paper they pick up they find a "smoking gun"!
Carl,
Well one could argue that no paper is perfect, given that they are all just approximating reality in some way. But alternatively auditing a paper by Lindzen, or Spencer, etc could certainly be a useful task for them :-)
"My first thought on a superficial glance at the paper was that it wasn't really that useful an analysis, as we already know that the models provide a decent hindcast of 20th century temps, so it's hardly surprising that looking at shorter trends will show the models agreeing on average over shorter trends too (since the full time series is merely the sum of shorter pieces). That leaves unasked the important question of how much the models have been tuned to reproduce the 20th century trend, and whether the recent divergence is the early signs of a problem or not. (Note that on the question of tuning, this is not even something that all modellers would have to be aware of, so honestly saying "we didn't do that" does not answer the question. But I digress.)"
Exactly - I don't understand what the paper could possibly be studying. What is left after you note that the vastly varying climate sensitivities didn't stop the models from doing okay over the last century?
As to the question of whether the models have been tuned: How is it possible to imagine that many very different models, with totally different climate sensitivities, could all pretty much match the last century of temperatures - without tuning to the temperature data? It doesn't even make sense to me.
I basically agree. However I also sympathise with people who argue that *they* did not tune their model - it's possible they just got lucky, and it's also possible that they did not consciously tune the model but built it as a new version implicitly learning from the previous version and keeping its behaviour largely compatible. But if someone does a trial run and gets a horrible result, I expect the model would be modified.
I have trouble imagining how to do it better. There is so little data, if your whole guide-post is to match very correlated temperature data, within a fifth of a degree or so. Mostly level, goes up for a while, levels off, goes up for a while, levels off this decade - that's about all there is to say. Not hard to do that, in so many different ways.
Is there some kind of general agreement that almost any conceivable regional variable is far too chaotic to model over a century, but global average temperature is not (conservation of energy, ignore ocean uptake...)? And we don't have any other global variable that might serve to augment our list of variables to emulate? If this is the way the problem is presented, it seems so hopeless. There just isn't very much data.
Oh, but the models also simulate large seasonal and spatial variation, in lots of detail, and the full 3d structure of ocean and atmosphere etc. Not perfectly, of course. But the sub-models that combine to form the whole are all tested in their own right for physical realism in various ways, before being stuck together. Global temp trend will be one of the last things checked, but my argument is that it is still (probably) checked prior to the model being frozen and used for the full suite of CMIP runs.
I understand, but are they expected to get those other things right for a century? Do we even have data for a century to compare with? I'm not saying the models aren't cool and well-designed; I'm just wondering how we tell how good they are at long-term prediction. I'm guessing - correct me - that no one even imagines that these other kinds of effects could be approximated for more than a short time scale.
Broadly speaking, these things haven't changed very much over a century but the models do get the major things roughly right like more warming at high latitudes and over land. I was really trying to say that the models simulate a whole lot of physical detail besides century-scale climate change, and this provides some evidence in suprort of them. After all, daily and seasonal cycles are the direct response of temporal changes in radiative input, not exactly the same as increasing CO2 but there's a coherence in the way these things are treated.
So to sum it all up i broad terms for the lazy: Where does this leave us in your opinion with respect to the long-term sensitivity (OK, just the Charney sensitivity)? Is it still around 2.5-3.0 C, or are you lingering a little more towards Nic Lewis-like values below 2 C?
I am now pretty convinced that there is a bit of a structural bias/error inherent to the 20th century calculations. Probably a low bias (not definitely). Basically, the effective sensitivity generally increases a bit with warming, so the first half generally shows a lower sensi than the second half. Adjusting for this would increase their ECS estimates a bit. Still needs work on quantifying this additional source of uncertainty, but there is strong evidence that there is something there...
Post a Comment