Monday, August 22, 2016

BlueSkiesResearch.org.uk: On the meaning of independence in climate science

At long last jules and I have managed to submit the written version of a talk that I have given (bits of) no fewer than four times over the last few years (at NCAR, UKMO/HC, Schloss Ringberg and EGU). It had to get re-written several times and sit at the back of my mind on long bike rides and runs before it became acceptably coherent (to us, at least). I’m curious as to what people will think of it – it seemed to go down ok at the talks but sometimes it’s hard to tell…

The topic is "independence" as it pertains to both the understanding of ensembles such as CMIPn, and also to constraints on climate sensitivity. Our main point overall is that if you want to talk about independence in any context, you really need to present a mathematical/statistical formalisation that relates directly to the standard probabilistic definition: events A and B are independent iff P(A,B) = P(A)P(B). That is, the probability of both A and B is the probability of A multiplied by the probability of B. This generalises to conditional independence: events A and B are conditionally independent given S iff P(A,B|S) = P(A|S)P(B|S). A more practically useful (but mathematically equivalent) formulation is that events A and B are conditionally independent given S iff P(A|S) = P(A|B,S) – that is, if the conditional probability of A given both S and B is the same as the conditional probability of A given S. What this means in practice to an individual researcher is that, starting from their (probabilistic) prediction of A given knowledge of S, conditional independence of A and B rests on whether knowledge of B, in addition to knowing S, does or does not change their prediction of A. While this is no more than elementary probability theory, it seems to be an intuitively attractive way of addressing the question, eg in the case where A and B are observational constraints on the equilibrium climate sensitivity S. A point we also make in the paper is that these P()s are fundamentally subjective things just as much as a Bayesian prior is – there is no way of validating what observations should be seen in the case where S takes a value different from the real world, this counterfactual can only really exist in our heads and not in reality. In practice we often use models for this, which are themselves subjective creations, and in the paper we present an example to show how independence and non-independence of constraints can be investigated in the context of a toy model.

As for the question of model independence, the notation may be easier to interpret if we change the symbols and write something like P(M1|T) = P(M1|M2,T). This equation asserts that the models M1 and M2 are conditionally independent given the truth T. This is essentially the foundation of the truth-centred approach, and it would be great if true, but clearly many analyses of the models in CMIP ensembles have shown that it is not reasonable. An alternative conditional independence formulation, which we think is more relevant and interesting, is whether models are independent, conditional on the distribution of models. We illustrate how this does seem to encapsulate much of the discussion of model similarity, in that models from different research centres seem independent whereas pairs of models from the same research centres do not, according to a fairly straightforward analysis of model similarity.

It is quite possible – likely – that some others will be able to improve on how we’ve tried to define independence, but our point was really to argue that in principle we must use a mathematical foundation in order to make any meaningful progress – and also observe that mathematical definitions do exist which seem to match at least some real-world usage reasonably well.

Thursday, August 18, 2016

Wickedly simple

I wasn't planning on writing anything about the Grundmann letter in Nature (paywalled of course, but you aren't missing much) as it seemed to be a pretty trivial “what about meeeeee” whine. ATTP put it best:
Both there and at the Onion, I haven't seen anything much to revise my previous jaundiced opinion as to the value of social science. I hope to be proved wrong but won't be holding my breath in the meantime.

One thing that I did find useful in the letter was a clear definition of what he thought it meant for a problem to be wicked: “Most importantly, wicked problems do not have a stopping rule. [...] Climate change does not have a stopping rule.”

I think that's pretty much flat out wrong. I know that people like to add on their favourite hobby horses of climate variability and social vulnerability etc, and some of the wording is vague, but the basic problem that has motivated and dominated the research agenda for many years is due to us pumping out millions of tonnes of fossil-fuel-derived CO2 into the atmosphere. Cut net emissions to about zero and the problem is solved. Sure, if you choose to define “climate change” sufficiently broadly, then you can always find something else to worry about, but stopping the global-scale experiment that we are currently performing would suffice for practical purposes. As they say, “good enough for govt work”.

For my next trick, I'll consider the problem of model independence.

Monday, August 01, 2016

Climate betting in the news again

Heard recently that Chris Hope eventually collected on his warming bet. Sounds like one of his protagonists had been dragging his heels, so that's great news. He'd already paid me my share out of the first part of his winnings. Chris wrote about it here a few days ago.

Nice article by Dana Nuccitelli in the Guardian today too:

And as it mentions, there's a session at the AGU Fall Meeting on closely related issues, which I've been invited to talk at. Will be good to see SF in December again, not been for a few years now. Probably 2012 was the last time. I'm also planning to submit something to this paleo session organised by PEN, not quite finished the abstract yet.