## Sunday, September 28, 2008

### Austin...where even the atheists worship on Sunday

A couple of weeks ago I went to the PMIP workshop in Estes Park. Since I was "in the area" (about as close as Tokyo to Beijing perhaps but we haven't yet popped to Beijing for tea) Michael Tobis and Charles Jackson arranged for me to visit Austin and the grand new climate science initiative at the University of Texas, although even now I am not quite sure if saying "climate" in the USA is really allowed. Even more excitingly, I arrived in Austin on Saturday so had a whole weekend to enjoy the city. For recreation I rented a carbon-forked vision of loveliness (but still their cheapest road bike) from the local vast glittering mecca of bicycleness. Of course I'd read doom laden apocalyptic warnings like this and this, and so was wondering if I'd get out alive. Yet I found that Austin surpassed all other US cities I've tried cycling in, including Boulder, the self-proclaimed cycling capital of America. I only cycled at the weekend, and I'm sure traffic is considerably heavier on weekdays, but in 7 hours of cycling I felt uniformly safe and unthreatened. For comparison, I usually get cross with a driver or two when riding my single bike in Japan, and urban Merseyside cycling was open warfare. The Austin drivers treated me with generosity, taking care at junctions and even changing lanes to overtake. There are a large number of wide, relatively clear bicycle lanes along the sides of many of the roads and I also had no trouble cycling on the roads that had no cycle lanes. Maybe I am just too comfortable jostling along with cars. The cycle paths are prone to stopping just before important junctions, but such is the way of the world, and things seemed nowhere near as foolish as in the UK. Finding my way was more of a challenge, and my compass was as invaluable in Austin as it is in Tokyo. Out of the city grid, there is some great cycling. Although the hills are too short, there are quite a lot of them, and there are trees and deer and stuff like that just a few miles from the city centre.

As I touristed around I found religion everywhere: the First Methodists, Second Baptists, Seventh Day Adventists, and I stopped for coffee in the Tenth St Arbucks. Not having stumbled upon an Episcopalian of any numeral, on Sunday morning I rejoined the minority bicycling sect instead. The roads were very clear. This seems to be because, on Sunday mornings in Austin, absolutely everyone is indoors expressing their beliefs, even the atheists, and the humanists. Although everyone else looked at me very very oddly when I remarked on how hilarious I found this, I suppose it is all the better for the cyclists. I was misunderstood on a number of other occasions too, those same odd looks appeared when I had been expecting a smile. Is this because Austin is Weird? I think it is rather because there is a tendency to take things literally. I even heard Biblical Literalism used in a positive sense, by some proponents of the teaching of Intelligent Design. To fend off all this literalism, perhaps it is British humour that should be taught in American schools.

## Thursday, September 11, 2008

### Hooray for Gordon

Just thought I'd take the opportunity for that blog title, since there are unlikely to be many more chances.

Back when I was living in the UK, Gordon was one of the most (if not the most) successful Chancellors ever. Now it seems that he is destined to be remembered primarily as one of the feeblest PMs. A sad but entirely predictable end to a political career (and yes I did predict it confidently, although maybe not in writing). Of course it is well-known that all political lives end in failure. I wonder why they never manage to quit while they are ahead. Did anyone ever get out at the top, leaving the masses desperate for more?

(Thinks for a minute.....ah, John Smith. That's taking things a little too far, though.)

"The government says its aim is to insulate every home in Britain by 2020 - and energy companies, councils and voluntary organisations will be making door-to-door visits in deprived areas to promote the scheme."Whether or not it is "enough" (and I'm sure plenty of people will be lining up to say it's inadequate), it is still a positive move. Although our old house will certainly not have cavity wall insulation by 2020, due to the fact that it has no cavity...

Back when I was living in the UK, Gordon was one of the most (if not the most) successful Chancellors ever. Now it seems that he is destined to be remembered primarily as one of the feeblest PMs. A sad but entirely predictable end to a political career (and yes I did predict it confidently, although maybe not in writing). Of course it is well-known that all political lives end in failure. I wonder why they never manage to quit while they are ahead. Did anyone ever get out at the top, leaving the masses desperate for more?

(Thinks for a minute.....ah, John Smith. That's taking things a little too far, though.)

Labels:
environment,
random

## Wednesday, September 10, 2008

### How not to combine multiple constraints...

Some time ago, I mentioned in passing a "howler" in press from CPDN. Since I was so sarcastic about it back then, it is only fair to provide an update. However, this post may be a bit long and technical for some...and things have changed enough since my previous statement, so much so that I probably wouldn't bother posting this but for my previous comment. But anyway, there isn't much else going on in climate science right now, so here goes...

The story starts way back with the IPCC AR4 drafts, actually. The very first semi-public version of this (ie the first version that people like me were allowed to see and comment on) attempted to address the issue of combining the different probabilistic estimates of climate sensitivity. They did this by averaging different pdfs that various authors have generated. It was immediately obvious to me (and possibly many others) that this was a completely invalid approach, and I said as much in my comments. In fact this episode - combined with the growing realisation that people were actually taking these outlandish "pdfs of climate sensitivity" seriously - is largely what prompted me to quickly write the "multiple constraints" paper. Anyway, this "averaging" result was removed from the next draft and I had hoped that our paper (which as far as I'm aware, no-one has argued against in the literature) might have put an end to that whole idea.

Fast forward on a few months and I saw an AGU presentation (by Reto Knutti) which appeared to again address the question of combining constraints. Its punchline was an analysis where where adding more observations appeared to result in an increase in uncertainty. Now, while it is certainly true that additional information can lead to an increase in uncertainty, such situations tend to be somewhat pathological and uncommon in practice, so I had a closer look. It only took a couple of emails back and with Reto to work out there was something badly wrong in what they had done - they had "combined" their constraints by averaging them, so I pointed out that this was invalid. Reto didn't reply to this point, and I assumed he had gone away and re-worked things. So, I was more than a little surprised that the next time I saw him speak, in the autumn of 2007, he presented the same result and mentioned that the paper (Sanderson et al) was now in press!

Time for a slight digression into why averaging the constraints is wrong. There is a standard theorem in probability theory that one always expects to learn from new information (and this somewhat woolly statement can be formalised in a rigorous manner). This general result crops up in all sorts of specific applications, and its correctness is not in doubt. Now, the sort of quadratic cost function that Sanderson et al use as their basic measure of model-data mismatch is pretty much ubiquitous in estimation, and it has a natural identification with the logarithm of the likelihood function corresponding to Gaussian uncertainties. It is no exaggeration to say that these quadratic log-likelihood cost functions underpin almost all practical probabilistic prediction and estimation in geosciences, all the way from a simple least-squares linear regression up to the advanced data assimilation methods used in modern numerical weather prediction. As a simple illustration, let's say we have an unknown variable x, and we make one observation o of it which has (Gaussian) observational uncertainty e. The resulting likelihood function P(o|x) is the Gaussian N(o,e), and taking the negative logarithm of this we reach the quadratic cost function 0.5*((x-o)/e)

The (normal, correct) cost function above corresponds to the Gaussian N((o1*e2

If, however, we use the average of the two single-obs cost functions, the resulting Gaussian is now N((o1*e2

So in summary, averaging constraints is obviously nonsense that has no possible probabilistic interpretation. I spent a lot of time late last year trying to explain this in simple terms to several of the authors of the Sanderson et al paper, and in response they gave every appearance of trying to argue that averaging was a valid approach, although they also accused me of reading too much into the paper. To be fair, they had not directly generated any posterior pdfs, and did not explicitly state that these cost functions were log-likelihoods, but given (a) the theoretical background I have briefly outlined above, and (b) the supposed increase in uncertainty as more observations were considered had been repeatedly presented as the main result of this work, I don't think my reading of it was unreasonable. Moreover, if they had really meant to disavow this interpretation, then it is puzzling that they continued to defend the methodology as valid. Myles Allen even challenged me to write a comment if I thought their paper was wrong, and given the potential impact of this erroneous "averaging the constraints" meme if it is widely adopted, I was originally tempted to do so. But I have just looked for the paper on-line, and am surprised (but pleased) to see several changes to the paper that have been made subsequent to our discussions. For starters, they have removed the most contentious claim - which was previously made in the abstract as well as the main text - that their cost function provides a constraint on climate sensitivity that weakens as more observations are used. In fact the paper now contains an entirely new paragraph that was not there before. This explicitly states that their results cannot be interpreted in terms of log-likelihoods. They could have gone a little further and admitted that the widths of their cost functions cannot be directly related to uncertainty in any way whatsoever, but I suppose that would have been rather embarrassing given that the explicit goal of the paper is to explore this issue. They have also added a little discussion of their reasoning behind the averaging method (which was not justified at all in the earlier version of the paper). Now it is explicitly presented as alternative to the "incorrect" alternative of adding the individual terms, which (as they state) is wrong if uncertainties are not independent. It is disappointing that they didn't find the space to mention here that the obvious alternative to adding the terms, if they are not independent, is just to use the full covariance matrix as I have outlined above. For while of course the addition of individual constraints is incorrect unless the uncertainties are judged to be independent, this may at least be a reasonable approximation if the dependencies are small (and it could in principle be biased either way), whereas their curious "averaging" method is guaranteed to be wrong for all possible covariance structures, with the resulting uncertainty guaranteed to be too wide. Indeed it is trivial to observe that averaging is algebraically equivalent to addition, but for the arbitrary and unjustifiable multiplicative scaling factor of sqrt(N) on the width of the implied joint likelihood function, where N is the number of individual terms.

I guess the authors would claim that these last-minute changes to the paper are nothing more than cosmetic - just a matter of explaining things in simple enough terms for people like me to understand, rather than representing any substantive modifications to their previous claims. (They certainly don't acknowledge my help in fixing the problems in the original version.) The real proof of the pudding will be in whether they and others continue to use this "averaging the constraints" meme...I will watch with interest to see if and how it spreads...

The story starts way back with the IPCC AR4 drafts, actually. The very first semi-public version of this (ie the first version that people like me were allowed to see and comment on) attempted to address the issue of combining the different probabilistic estimates of climate sensitivity. They did this by averaging different pdfs that various authors have generated. It was immediately obvious to me (and possibly many others) that this was a completely invalid approach, and I said as much in my comments. In fact this episode - combined with the growing realisation that people were actually taking these outlandish "pdfs of climate sensitivity" seriously - is largely what prompted me to quickly write the "multiple constraints" paper. Anyway, this "averaging" result was removed from the next draft and I had hoped that our paper (which as far as I'm aware, no-one has argued against in the literature) might have put an end to that whole idea.

Fast forward on a few months and I saw an AGU presentation (by Reto Knutti) which appeared to again address the question of combining constraints. Its punchline was an analysis where where adding more observations appeared to result in an increase in uncertainty. Now, while it is certainly true that additional information can lead to an increase in uncertainty, such situations tend to be somewhat pathological and uncommon in practice, so I had a closer look. It only took a couple of emails back and with Reto to work out there was something badly wrong in what they had done - they had "combined" their constraints by averaging them, so I pointed out that this was invalid. Reto didn't reply to this point, and I assumed he had gone away and re-worked things. So, I was more than a little surprised that the next time I saw him speak, in the autumn of 2007, he presented the same result and mentioned that the paper (Sanderson et al) was now in press!

Time for a slight digression into why averaging the constraints is wrong. There is a standard theorem in probability theory that one always expects to learn from new information (and this somewhat woolly statement can be formalised in a rigorous manner). This general result crops up in all sorts of specific applications, and its correctness is not in doubt. Now, the sort of quadratic cost function that Sanderson et al use as their basic measure of model-data mismatch is pretty much ubiquitous in estimation, and it has a natural identification with the logarithm of the likelihood function corresponding to Gaussian uncertainties. It is no exaggeration to say that these quadratic log-likelihood cost functions underpin almost all practical probabilistic prediction and estimation in geosciences, all the way from a simple least-squares linear regression up to the advanced data assimilation methods used in modern numerical weather prediction. As a simple illustration, let's say we have an unknown variable x, and we make one observation o of it which has (Gaussian) observational uncertainty e. The resulting likelihood function P(o|x) is the Gaussian N(o,e), and taking the negative logarithm of this we reach the quadratic cost function 0.5*((x-o)/e)

^{2}. If we have more than one observation, things are little more complicated because we need to account not only for the uncertainties on each observation, but also the covariance of these uncertainties. The resulting multivariate Gaussian likelihood N(O,E) naturally gives rise to the cost function 0.5*(X-O)^{T}E^{-1}(X-O) where X and O are now vectors of the estimated and observed variables, and E is the covariance matrix. In the simple case where we have two observations of a single variable and the covariance of the errors E is diagonal, this simplifies to 0.5*[((x-o1)/e1)^{2}+ ((x-o2)/e2)^{2}] - and this is the direct sum of the cost functions that arise from taking each observation individually. And whatever the form of the covariance matrix E, the log-likelihood is never equivalent to 0.25*[((x-o1)/e1)^{2}+ ((x-o2)/e2)^{2}], which is the result of the "averaging the constraints" procedure that Sanderson et al invented.The (normal, correct) cost function above corresponds to the Gaussian N((o1*e2

^{2}+o2*e1^{2})/(e1^{2}+e2^{2}), sqrt(e1^{2}*e2^{2}/(e1^{2}+e2^{2})), this being the well-known formula for the optimal interpolation of two uncertain estimates. It is easy to see that the width of this Gaussian is lower than for either of the single-observation Gaussians, since sqrt(e1^{2}*e2^{2}/(e1^{2}+e2^{2})) is less than the smaller of e1 and e2. (The same result holds if we use a full covariance matrix E with off-diagonal elements, but it's not as easy to show that within Blogger's limited maths capability!)If, however, we use the average of the two single-obs cost functions, the resulting Gaussian is now N((o1*e2

^{2}+o2*e1^{2})/(e1^{2}+e2^{2}),sqrt(2*e1^{2}*e2^{2}/(e1^{2}+e2^{2})) and it is immediate that its width, sqrt(2*e1^{2}*e2^{2}/(e1^{2}+e2^{2}), lies between e1 and e2. So, if we start with the more accurate observation, and then "combine" it with a less accurate one through this "averaging" methdology, then the final answer will always have greater uncertainty than we started with, which is contrary to the well-established theorem I mentioned previously. Effectively, this method is asserting that a weak observation with wide uncertainty should cause us to forget what we already knew!So in summary, averaging constraints is obviously nonsense that has no possible probabilistic interpretation. I spent a lot of time late last year trying to explain this in simple terms to several of the authors of the Sanderson et al paper, and in response they gave every appearance of trying to argue that averaging was a valid approach, although they also accused me of reading too much into the paper. To be fair, they had not directly generated any posterior pdfs, and did not explicitly state that these cost functions were log-likelihoods, but given (a) the theoretical background I have briefly outlined above, and (b) the supposed increase in uncertainty as more observations were considered had been repeatedly presented as the main result of this work, I don't think my reading of it was unreasonable. Moreover, if they had really meant to disavow this interpretation, then it is puzzling that they continued to defend the methodology as valid. Myles Allen even challenged me to write a comment if I thought their paper was wrong, and given the potential impact of this erroneous "averaging the constraints" meme if it is widely adopted, I was originally tempted to do so. But I have just looked for the paper on-line, and am surprised (but pleased) to see several changes to the paper that have been made subsequent to our discussions. For starters, they have removed the most contentious claim - which was previously made in the abstract as well as the main text - that their cost function provides a constraint on climate sensitivity that weakens as more observations are used. In fact the paper now contains an entirely new paragraph that was not there before. This explicitly states that their results cannot be interpreted in terms of log-likelihoods. They could have gone a little further and admitted that the widths of their cost functions cannot be directly related to uncertainty in any way whatsoever, but I suppose that would have been rather embarrassing given that the explicit goal of the paper is to explore this issue. They have also added a little discussion of their reasoning behind the averaging method (which was not justified at all in the earlier version of the paper). Now it is explicitly presented as alternative to the "incorrect" alternative of adding the individual terms, which (as they state) is wrong if uncertainties are not independent. It is disappointing that they didn't find the space to mention here that the obvious alternative to adding the terms, if they are not independent, is just to use the full covariance matrix as I have outlined above. For while of course the addition of individual constraints is incorrect unless the uncertainties are judged to be independent, this may at least be a reasonable approximation if the dependencies are small (and it could in principle be biased either way), whereas their curious "averaging" method is guaranteed to be wrong for all possible covariance structures, with the resulting uncertainty guaranteed to be too wide. Indeed it is trivial to observe that averaging is algebraically equivalent to addition, but for the arbitrary and unjustifiable multiplicative scaling factor of sqrt(N) on the width of the implied joint likelihood function, where N is the number of individual terms.

I guess the authors would claim that these last-minute changes to the paper are nothing more than cosmetic - just a matter of explaining things in simple enough terms for people like me to understand, rather than representing any substantive modifications to their previous claims. (They certainly don't acknowledge my help in fixing the problems in the original version.) The real proof of the pudding will be in whether they and others continue to use this "averaging the constraints" meme...I will watch with interest to see if and how it spreads...

Labels:
climate science,
climate sensitivity

## Wednesday, September 03, 2008

### Snow in August

Thought I had blogged this ages ago, but it seems to be still at the draft stage...

Snow fell on Mt Fuji Aug 9 at earliest time ever

However, it seems that this new record is due to an arbitrary decision to redefine some hail as snow. It has still been unseasonally wet and cold for the latter part of the summer - which I would have been quite happy about but for the fact that we were hoping to have another trip into the mountains. I'm certainly glad we made the effort earlier.

Snow fell on Mt Fuji Aug 9 at earliest time ever

However, it seems that this new record is due to an arbitrary decision to redefine some hail as snow. It has still been unseasonally wet and cold for the latter part of the summer - which I would have been quite happy about but for the fact that we were hoping to have another trip into the mountains. I'm certainly glad we made the effort earlier.

### Japanese murder investigators fooled by life-sized sex doll

Can't...resist...blogging...

What they don't mention in this story is that by the time the police realised it was a doll, it had already confessed to two charges of shoplifting and one of child neglect.

It is now rumoured to be considering standing as a candidate in the forthcoming election for the new Prime Minister.

Police in Japan have been left red-faced by an apparent murder that turned out to be an unusual case of mistaken identity.

What they don't mention in this story is that by the time the police realised it was a doll, it had already confessed to two charges of shoplifting and one of child neglect.

It is now rumoured to be considering standing as a candidate in the forthcoming election for the new Prime Minister.

### Corbyn again

The August data are in, and once again it is 2 out of 2 for Piers! The daily max temperature was 1.2C below average, covered by his range of 0.3 - 1.3C colder than average, and rainfall at 165% was also inside his range of 125%-175%.

So he's up to 8 out of 14, which means the p-value is still below 5%, but it is heading in the right direction. There is no sign of the September forecast yet. The August one only appeared on the 8th, by which time it was already well set to be very a wet month with cold days, but of course he claims that this forecast was really made some time previously...

So he's up to 8 out of 14, which means the p-value is still below 5%, but it is heading in the right direction. There is no sign of the September forecast yet. The August one only appeared on the 8th, by which time it was already well set to be very a wet month with cold days, but of course he claims that this forecast was really made some time previously...

Subscribe to:
Posts (Atom)