Wednesday, January 18, 2006

Prometheus: Myanna Lahsen's Latest Paper on Climate Models Archives

Roger Pielke Jnr has posted up excerpts from an interesting paper by Myanna Lahsen on climate modellers and the "trough of uncertainty". She paints a persuasive picture of modellers sometimes having an unhealthy level of belief in their models, and overselling their confidence in their results for a number of reasons. I'm sure there's some truth in that, at least as a couterweight to the existing paradigm that those closest to model-building are most aware of the warts (hence "trough", with the modellers and alienated critics being most sceptical, and the poor users being overly credulous).

There are also, of course, many users in the prediction end of the field for whom the models are explicitly considered as being merely an uncertain source of information about possible futures, and nowhere near to a being a crystal ball (though we may still use "the ocean" as shorthand for "the modelled ocean"). Moreoever, assessments like the IPCC necessarily spread the ball of uncertainty to include a wide range of perspectives (whether or not you think it should be even broader, there is no question that it is much wider than any one person's view). It is also worth mentioning one recent notable occasion when disagreement between models and data was essentially resolved in favour of the models. It would not, I believe, be at all reasonable to conclude from her work that all climate science is massively oversold (I can hear the septics sharpening their pens), but a healthy dose of rational scepticism is generally useful.

One thing Roger didn't feature is her comments about meteorologists, which may be interesting to those who have noticed the unreasonable level of hostility that certain American State Climatologists have shown towards climate modellling:
Synoptically trained empirical meteorologists have particular motivation to resent models. Their methods and lines of work were in important part replaced by global numerical models. The environmental concern about human-induced climate change, and the associated politics, also favored the GCMs and those working with them. The applied aspect of these meteorologists’ work was thus being taken over by numerical weather forecasting, pushing them in the direction of basic research. Their comments should be understood as potentially interested instances of boundary-work (Gieryn, 1995) whereby they, as a competing scientific group, seek to promote the authority of their own lines of research in competition with GCMs. This placed them at a competitive disadvantage when national funding priorities changed in favor of research with obvious social benefits, whereas GCM modeling seemed relevant to predicting future manifestations of human-induced climate change.
The emergence of numerical techniques also represented a loss in epistemic status as well as funding for the empirical meteorologists. So called ‘objective’ numerical methods resulted in the demotion and relabeling of their more qualitative approach as ‘subjective’, an unattractive label in the context of a cultural preference for ‘hard’ science within the scientific community.
Read the whole paper - with a sceptical mind :-)

5 comments:

Myanna Lahsen said...

I'm happy to see discussion of my paper.

I want to point persons on this blog to the first comment to my paper posted on Prometheus. I think it highlights something some may be inclined to overlook when reading the paper, especially if they are already critical of models. I write this because my study can be used as a cheap shot against modelers, wherefore it is important to keep the above things in focus as well.

The person who wrote that first comment described my case study of modelers as "an important example of larger issue both in science and in general perceptions of the world," noting that "There is a real human tendency for people to confuse their 'models' and 'constructs' of the world with reality. This is true of economics, psychology and the hard sciences." He also, correctly, points out that quotes in my paper indicate that modelers as a whole "are very aware of the pitfalls, and to some extent the community is partially self correcting. For example: "It is easy to get a bad name as a modeler, among both theoreticians and observational people, by running experiments and seeing something in the model and publishing the result. And pretending to believe what your model gives - or, even, really believing it! ..."

(for the whole comment, see http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/000675myanna_lahsens_late.html#comments )

With all due respect, I would therefore also prefer to not have my argument summarized as showing that modelers have "an unhealthy belief" in their models. I may have invited the interpretation of modelers’ relationship to their models as “unhealthy” because I at one point in the paper write that modelers “do not necessarily have a consistent, ‘healthy skepticism’” in relationship to their models. Of course, the words "do not necessarily" adds important qualification and nuance to my argument, though the latter understandably might get lost in the transmission process. What needs to be kept in mind is that the same can be said for the use of models - including theoretical frameworks - in general. I make the point about potential lack of critical distance in the context of refuting MacKenzie’s uncertainty trough model for the distribution of uncertainty around scientific productions.

What I hope people walk away with from reading the paper is that "reality" is indeed a difficult thing to know, and that no one - including more empirically oriented climate scientists - have privileged access to it: all renditions of reality are mediated by the methods we use to understand it, and by our conceptual frameworks. It may be that simulations have a particularly persuasive hold on the imagination, a notion I entertain in the paper.

Something that happens a lot in the US politics around climate science and climate change is that one side suggests that they are objective and know reality, by contrast to their opponents. That rhetoric prevails on both sides, and I can see that my paper might be fodder to climate skeptics in that war.

In anthropology, we tend to think that the most robust understanding emerges from considering a plethora of different viewpoints, each of which contributes one part of the whole, and each of which also needs to be scrutinized and understood as rooted in particularity of perspective.

Cheers,
Myanna Lahsen

EliRabett said...

We also have examples of folk who fall in love with their data (analysis). The issue is not a single modeler or MSU data set analyst, the problem is if the larger community falls in love.

Everyone is in love with what they do. Their ardor is restrained by the community around them. Given that people work amidst small to medium size groups of others who are their friends and colleagues, this points up the importance of those trips to NZ and other fine places where one's ideas have to play in the open field.

Alternatively one sees the value of pissing off everyone in the immediate vacinity so they will be sceptical of all your claims.

James Annan said...

Myhanna,

Thanks for the follow-up. But please note that just as your original words included the qualifier that modellers "do not necessarily have a consistent, 'healthy scepticism'", so did my summary of "modellers sometimes having an unhealthy level of belief". I really don't see that my words can be considered as a misrepresentation, and I'm certainly not intending any cheap shots myself.

James Annan said...

Alternatively one sees the value of pissing off everyone in the immediate vacinity so they will be sceptical of all your claims.

Brilliant, Eli! You've found the ideal post-hoc rationalisation for all I've been doing all these years! I'm just making sure my science is robust!

Do you think I can claim it was deliberate?

Myanna Lahsen said...

Fair enough, James.

I just felt a need to make that point, before the "sometimes" would be lost in other summaries. It tends to work like that in a politicized playing field.

Cheers,
Myanna