Last weekend another visit to far off lands. This time to visit James' auld mither an faither.
--
Posted By jules to jules' pics at 8/31/2010 05:28:00 AM
Tuesday, August 31, 2010
Friday, August 27, 2010
[jules' pics] 8/27/2010 01:49:00 AM
Maximum Entropy day was yesterday. It is fortunate then that the least important part of making a pilgrimage to Lourdes is the actual cricket.
--
Posted By jules to jules' pics at 8/27/2010 01:49:00 AM
Wednesday, August 25, 2010
[jules' pics] 8/25/2010 03:22:00 AM
More about the program; this morning is all about tipping punts. Sorry..I mean Timmy points. Timmy himself gave a fun overview talk about them on Monday, but now its gone all nitty gritty. Still - must soldier on - the workshop dinner is tonight.
--
Posted By jules to jules' pics at 8/25/2010 03:22:00 AM
Tuesday, August 24, 2010
jules pics - robin
All getting very Britishy now...we don't get these back home. I suppose this might be a young robin coming into adult plumage, since it has a brown speckled head... probably blog readers know better and I looked forward to being corrected.
Having said that about the Britishness, this was shot in Kent, which is a bit of a rum place. While it seems to be about the only place in the UK which actually enjoys the kind of summers assumed in English children's books (that is the kind that is full of ripe forest fruits and golden meadows, and during which one may venture outdoors without a big jumper and a raincoat), Kent also has fleets of parakeets and big snakes!
Monday, August 23, 2010
Mathematical and Statistical Approaches to Climate Modelling and Prediction
A commenter worked out where we were and asked about the programme. I haven't had anything to say so far because although we have now been here for more than a week, it was pretty quiet at first, and we were mostly spending our time on some fairly routine work we had brought with us. This week, however, is the first workshop, and it is rather busier.
My first thoughts on reading the blurb (back when we were first invited last year) was that it seemed to have a slightly split personality to me, with two components that have only tenuous links. The second component is the one that directly interests me, as it concerns the development and use of probabilistic methods for climate prediction (which in context and based on attendees primarily means the O(100y) problem). The first component, however, which is the main topic of this opening workshop, is the use of stochastic sub-grid physics parameterisations, which can modify (and improve) the behaviour of weather and climate models in various ways. Tim Palmer has been pushing this for some time and it's clear that he is on to something as far as improving short and medium term weather forecasting. However, he presented this as a grand plan for replacing the ensemble of CMIP models, but it seems immediately obvious to me that the stochastic physics approach does not begin to address the sort of uncertainties in the equilibrium climate state and response that dominate the climate prediction problem on the century time scale. It's just a better parameterisation, but still in principle a single parameterisation which will give rise to a single climate and single climate sensitivity etc, as the uncertainty the stochastic part introduces will generally be negligible over long time scales (coincidentally, I wrote a short and trivial paper on this way back in the mists of time). In order to generate climatologically different models, we would need different probabilistic schemes, not just a different sequence of samples from the same scheme. I asked him about this after his talk and was not convinced by his response.
There was also a talk on maximum entropy, which confirmed my suspicion that I should not attempt to learn anything about maximum entropy. I consider that an hour well spent!
My first thoughts on reading the blurb (back when we were first invited last year) was that it seemed to have a slightly split personality to me, with two components that have only tenuous links. The second component is the one that directly interests me, as it concerns the development and use of probabilistic methods for climate prediction (which in context and based on attendees primarily means the O(100y) problem). The first component, however, which is the main topic of this opening workshop, is the use of stochastic sub-grid physics parameterisations, which can modify (and improve) the behaviour of weather and climate models in various ways. Tim Palmer has been pushing this for some time and it's clear that he is on to something as far as improving short and medium term weather forecasting. However, he presented this as a grand plan for replacing the ensemble of CMIP models, but it seems immediately obvious to me that the stochastic physics approach does not begin to address the sort of uncertainties in the equilibrium climate state and response that dominate the climate prediction problem on the century time scale. It's just a better parameterisation, but still in principle a single parameterisation which will give rise to a single climate and single climate sensitivity etc, as the uncertainty the stochastic part introduces will generally be negligible over long time scales (coincidentally, I wrote a short and trivial paper on this way back in the mists of time). In order to generate climatologically different models, we would need different probabilistic schemes, not just a different sequence of samples from the same scheme. I asked him about this after his talk and was not convinced by his response.
There was also a talk on maximum entropy, which confirmed my suspicion that I should not attempt to learn anything about maximum entropy. I consider that an hour well spent!
More on the multi-model mean
Well the mystery surrounding the origins of this equation has been at least partially resolved. It is clearly presented in Stephenson and Doblas-Reyes "Statistical methods for interpreting Monte Carlo ensemble forecasts" Tellus 2000 (as the first author pointed out to me last week), who attribute the result to Epstein 1969 and Leith 1974. The attribution seems a little unclear to me, as these earlier papers both present a large number of equations relating to the evolution of forecast ensembles of specific systems, whereas this identity is fully general irrespective of the ensemble generation. However it is not in doubt that it is well known by lots of people and dates to the origins of ensemble methods in weather prediction, even though it has perhaps been subsequently forgotten by some (and apparently never known in climate science, or indeed numerous other fields).
[jules' pics] 8/23/2010 05:50:00 AM
Anonymous suggests we blog the program. So here it is.
Prom 50, 22nd August 2010:
Mozart: The Magic Flute - Overture
Bartók: Piano Concerto No.3
Bartók: Cantata profana
Haydn: Symphony No.102 in B flat major
Kamakura is an hour on the train from Tokyo, Cambridge the same from London, and just like at home, the train whizzed us effortless home from the night-time fun in the big city - well, not quite as comfortably, but it was on time.
--
Posted By jules to jules' pics at 8/23/2010 05:50:00 AM
Friday, August 20, 2010
[jules' pics] 8/20/2010 04:27:00 AM
A brief break between rain showers in Cambridge.
--
Posted By jules to jules' pics at 8/20/2010 04:27:00 AM
Thursday, August 19, 2010
[jules' pics] 8/19/2010 02:31:00 AM
jules tries to blend in... hopefully redressing the Daily Mail imbalance.
--
Posted By jules to jules' pics at 8/19/2010 02:31:00 AM
Wednesday, August 18, 2010
[jules' pics] 8/18/2010 01:33:00 AM
Nick Barnes is to be congratulated for working out that we are in Cambridge.
Now that's out of the way perhaps I can post some nice photos of Cambers. Here's a bauble wot some old mate of my father-in-law recently stuck up outside Corpus Christi College, which, coincidentally is also where I did my PhD. The bauble is part of a clock which has a beetle on the top of if that is almost as large as some of the ones we get in Japan, but the glass in front of the clock is very reflective so multiple reflections seemed the way to go photographically. That's the front of King's College that you can see behind James.
--
Posted By jules to jules' pics at 8/18/2010 01:33:00 AM
Tuesday, August 17, 2010
[jules' pics] How to Punt Cleverly
[People with slow internets may find this difficult to see. I find that once the animated gif has done one run through it speeds up to the intended speed.]
While James can add up adequately, he can actually punt cleverly, although I am not sure if he can do both at the same time! The real reason for adopting this style is, of course, so one can drink Pimms at the same time, but it was only 10am, and we are old and not strong at alcohol...
--
Posted By jules to jules' pics at 8/16/2010 12:58:00 PM
Aircon!
Guthrie asks if anything else if different since we were in the UK. Aircon!
While it is very pleasing to us Japanese that the silly English (oh no I've given away the country we are in!) are buying our beautiful, efficient aircon, they really need to learn to leave them switched off until it is at least warm. The one in the photo (Toshiba - hurrah!) isn't actually on, thankfully, or we wouldn't have stayed so long in the pub, but we have several times been in coffee shops, where it is cold outside (under 20C) and even colder in the shop itself, such that we didn't really enjoy having to hang about and drink the coffee. This seems to me to be the opposite of adapting to temperature increase; spending more money on electricity to keep the temperature lower than it has ever been. Seems the Japanese are doing all the "gaman" on this; going to work almost naked and keeping their aircon at 28C. Of course, it is true that a lot of electricity is used in Japan on aircon - it hit 38C in Tokyo yesterday, which means that even aircon set to 28C would be struggling.
Monday, August 16, 2010
"Health and safety of the buildings"
I'd read about things like this on the intertubes, but never expected it would actually happen to us. We were wandering around the town centre of XXX at the weekend, and walked into a shopping centre/mall. Jules had her camera and decided to take a picture (she's keen on that, you may have noticed). Shortly after completing the snap, a security guard came up and told us we weren't allowed to do that. Why not, Jules asked (not being in Japan any more). "Health and safety of the buildings" he replied. We laughed and told him that was obviously rubbish, and then he said it was because of security. After all, any self-respecting thief will take pictures from the outside of the shop with a big fat DSLR in full view of the CCTV systems, rather than surreptitiously take snaps with a phone or concealed camera...
I assumed at the time that as shopping centres are generally private land, the owners do actually have the right to forbid photography (technically, they do this by withdrawing the implied permission of access, making the photographer a trespasser). So I didn't argue any further. However, I have now seen that although it's not actually a public right of way, there used to be one there before the mall was built, and there is some sort of bylaw guaranteeing public access during daytime, with provisions for public order and obstruction but nothing relating to photography being banned. So now I'm disappointed I didn't stand up for my rights.
I assumed at the time that as shopping centres are generally private land, the owners do actually have the right to forbid photography (technically, they do this by withdrawing the implied permission of access, making the photographer a trespasser). So I didn't argue any further. However, I have now seen that although it's not actually a public right of way, there used to be one there before the mall was built, and there is some sort of bylaw guaranteeing public access during daytime, with provisions for public order and obstruction but nothing relating to photography being banned. So now I'm disappointed I didn't stand up for my rights.
[jules' pics] 8/16/2010 01:16:00 AM
After a stopover in London we headed to the land of the Tesco Mega Temple. Wow. Having said that, I have seen better beer selections...
--
Posted By jules to jules' pics at 8/16/2010 01:16:00 AM
Saturday, August 14, 2010
[jules' pics] 8/14/2010 04:07:00 AM
There's a traditional oven in our apartment*! Furthermore, cake ingredients are not only readily available but are cheap at the local Tescos mega temple.
Weeeirdd! These yellow butterish products said only what they are not. For baking, I had to be sure it was margarine, so I avoided this array of dubiousness and eventually found a slab of something called "Stork, Perfect for pastry", which said it was 75% vegetable fat. Is that enough? Brownies look OK, fortunately. But what happened to good old margarine?!
*In Japan our only oven is a combi microwave that we brought over from the UK with us. Mostly only rich people have real ovens in Japan.
--
Posted By jules to jules' pics at 8/14/2010 04:07:00 AM
Once more into the breach
Hmm...I've used that title before. Anyway, I've run out of excuses and have been tasked by my manager with addressing some of the issues people have raised in comments, while she addresses the current shortage of chocolate brownies in our apartment. Thanks to those who have already stepped in and provided some answers and rebuttals.
Rather than just working through the comments in detail, I think it may be more sensible to firstly re-iterate some basic points underlying the interpretation of model ensembles, as this seems to be the basis of the disagreement. I'm not going to delve into the fine details of the statistical analyses but rather consider the broad issue of what an ensemble of model results can reasonably be expected to represent.
I'm actually finding it difficult to criticise MMH too strongly - it's not that they are correct, on the contrary it is clear that their analysis is wrong-headed and fundamentally irrelevant - but rather that so many climate scientists have also got confused over this. MMH have compared the obs to the ensemble mean and found that they differ. As I said before, big deal, no-one in their right mind would expect them to match anyway. But the "IPCC Experts"TM did effectively endorse such an interpretation of the ensemble in their recent Expert Guidance (hmmm...dare I call it "The Dummies Guide to Ensembles"? Better not go there, I'm sure I've done enough damage already.) So clearly they are also rather confused.
Of course it would be great if it were true. No more worries over climate sensitivity, for example. Given the current model spread of about 3±1.5, all we need to do is put together 10 models into an ensemble and the range shrinks to 3±0.5. 100 models would give us 3±0.15. Who needs all those pesky new observations and theories anyway? Just write some more code and the answer will pop out. Sadly, real life doesn't work this way.
To put it in a nutshell, there is simply no theoretical, philosophical, or practical basis for the hope that the ensemble mean may coincide with the truth. I have demonstrated that the mean is always a better estimate - in terms of having a lower RMS error - than most of the constituent models, and explained why it is often better than all of the models - but that doesn't make it the truth. In fact irrespective of how good the models are, the mean of the models is not a plausible climate simulation at all in many respects. The mean is a mathematical object that doesn't even look like a model - it has far too little variability in both time and space, and numerous other completely unrealistic properties. This idea is completely routine and well-known, but Reto Knutti sometimes presents a nice analogy demonstrating this (I see he attributes the idea to Doug Nychka):
It's a set of 16 photos of aeroplanes (which take the place of models aiming to simulate a hypothetical "real plane") and the "average photo" composite of the images. The planes all have 2 wings and recognisable shapes. The average plane is a vague splodge, with no recognisable shape. It simply doesn't look like a plane at all. Surely no-one in their right mind would be surprised if the real plane fails to end up looking precisely like this "average plane". Except MMH and perhaps the IPCC Experts. Sorry, but however much I would like to just have a go at MMH I can't get around the fact that the IPCC Experts explicitly endorsed this idea.
In our GRL paper we wrote:
"This truth-centred paradigm appears to have arisen as a post-hoc interpretation of the ad-hoc weighting procedure known as ‘‘Reliability Ensemble Averaging’’ or REA"
and
"We suggest that in place of the truth-centred approach, future research into the use of the CMIP3 and other multi-model ensembles of opportunity should be based on the paradigm of a statistically indistinguishable ensemble, as this is both intuitively plausible and reasonably compatible with observational evidence."
which was intended as a strong condemnation, but perhaps it was too cryptic, or perhaps the IPCC Experts simply weren't prepared to take our word for it. It seems that Jules and I will have to keep writing about this in a variety of ways until the idea sinks in. The belief that an ensemble should or even could be centred on an unknown truth is a complete nonsense. End of story. There is really nothing to debate here.
Rather than just working through the comments in detail, I think it may be more sensible to firstly re-iterate some basic points underlying the interpretation of model ensembles, as this seems to be the basis of the disagreement. I'm not going to delve into the fine details of the statistical analyses but rather consider the broad issue of what an ensemble of model results can reasonably be expected to represent.
I'm actually finding it difficult to criticise MMH too strongly - it's not that they are correct, on the contrary it is clear that their analysis is wrong-headed and fundamentally irrelevant - but rather that so many climate scientists have also got confused over this. MMH have compared the obs to the ensemble mean and found that they differ. As I said before, big deal, no-one in their right mind would expect them to match anyway. But the "IPCC Experts"TM did effectively endorse such an interpretation of the ensemble in their recent Expert Guidance (hmmm...dare I call it "The Dummies Guide to Ensembles"? Better not go there, I'm sure I've done enough damage already.) So clearly they are also rather confused.
Of course it would be great if it were true. No more worries over climate sensitivity, for example. Given the current model spread of about 3±1.5, all we need to do is put together 10 models into an ensemble and the range shrinks to 3±0.5. 100 models would give us 3±0.15. Who needs all those pesky new observations and theories anyway? Just write some more code and the answer will pop out. Sadly, real life doesn't work this way.
To put it in a nutshell, there is simply no theoretical, philosophical, or practical basis for the hope that the ensemble mean may coincide with the truth. I have demonstrated that the mean is always a better estimate - in terms of having a lower RMS error - than most of the constituent models, and explained why it is often better than all of the models - but that doesn't make it the truth. In fact irrespective of how good the models are, the mean of the models is not a plausible climate simulation at all in many respects. The mean is a mathematical object that doesn't even look like a model - it has far too little variability in both time and space, and numerous other completely unrealistic properties. This idea is completely routine and well-known, but Reto Knutti sometimes presents a nice analogy demonstrating this (I see he attributes the idea to Doug Nychka):
It's a set of 16 photos of aeroplanes (which take the place of models aiming to simulate a hypothetical "real plane") and the "average photo" composite of the images. The planes all have 2 wings and recognisable shapes. The average plane is a vague splodge, with no recognisable shape. It simply doesn't look like a plane at all. Surely no-one in their right mind would be surprised if the real plane fails to end up looking precisely like this "average plane". Except MMH and perhaps the IPCC Experts. Sorry, but however much I would like to just have a go at MMH I can't get around the fact that the IPCC Experts explicitly endorsed this idea.
In our GRL paper we wrote:
"This truth-centred paradigm appears to have arisen as a post-hoc interpretation of the ad-hoc weighting procedure known as ‘‘Reliability Ensemble Averaging’’ or REA"
and
"We suggest that in place of the truth-centred approach, future research into the use of the CMIP3 and other multi-model ensembles of opportunity should be based on the paradigm of a statistically indistinguishable ensemble, as this is both intuitively plausible and reasonably compatible with observational evidence."
which was intended as a strong condemnation, but perhaps it was too cryptic, or perhaps the IPCC Experts simply weren't prepared to take our word for it. It seems that Jules and I will have to keep writing about this in a variety of ways until the idea sinks in. The belief that an ensemble should or even could be centred on an unknown truth is a complete nonsense. End of story. There is really nothing to debate here.
Boring and useless
A little light relief before I wade in (though the comments and blog posts have been light relief enough already).
Just before leaving, I was told I had an additional ¥750,000 in my budget. Which had to be spent (at least allocated) by yesterday. Don't get too excited, it's only about $9000 and is to be spread amongst various colleagues, but it's still better than a slap in the face with some raw sushi, as they say in Japan. Anyway, my existing iMac is about 3 years old, so clearly I'm first in line for a replacement. But even with extra software, that hardly gets through half of it (I don't need a really powerful machine, it's not for computing). I thought that some could usefully be put towards paper charges, given that we have several unbudgeted manuscripts working their way through the system (they can't all be rejected, surely). On top of that, I had my eyes on a bunch of interesting books that I could learn from osmotically if not actually read.
Oops, we can't pay for paper charges with this money, because...we couldn't pay for paper charges last year. Oh, well, we thought, let's really scrape the bottom of the barrel and waste the money on some iPads instead. A cubicle-neighbour bought one himself recently (out of his own pocket) and apparently they are great pdf readers for cramped trains and the like.
Nope, JAMSTEC won't let us buy iPads becaue they are not allowed. Oh, and even books are banned too. Why? I've learnt there is rarely any point even asking, so I didn't bother.
So, I have about $4000 burning a hole in my pocket. We can't spend it on anything useful like books or paper charges, or anything fun like iPads. I guess that leaves boring and useless stuff. I have one last suggestion of an electric umbrella-dryer (don't laugh, we have seen only in the foyer of another lab, and can guess why it was bought...). Failing that, I could do what another now ex-colleague did a few years ago and buy another desktop computer and simply leave it in its box. In fact that is the machine I am currently using, as I snaffled it when they left last year (still unused).
Has anyone else got any other ideas?
Having only been in Japan during austere years of stagnation and budget cuts, I can hardly imagine what the bubble years must have been like...
Just before leaving, I was told I had an additional ¥750,000 in my budget. Which had to be spent (at least allocated) by yesterday. Don't get too excited, it's only about $9000 and is to be spread amongst various colleagues, but it's still better than a slap in the face with some raw sushi, as they say in Japan. Anyway, my existing iMac is about 3 years old, so clearly I'm first in line for a replacement. But even with extra software, that hardly gets through half of it (I don't need a really powerful machine, it's not for computing). I thought that some could usefully be put towards paper charges, given that we have several unbudgeted manuscripts working their way through the system (they can't all be rejected, surely). On top of that, I had my eyes on a bunch of interesting books that I could learn from osmotically if not actually read.
Oops, we can't pay for paper charges with this money, because...we couldn't pay for paper charges last year. Oh, well, we thought, let's really scrape the bottom of the barrel and waste the money on some iPads instead. A cubicle-neighbour bought one himself recently (out of his own pocket) and apparently they are great pdf readers for cramped trains and the like.
Nope, JAMSTEC won't let us buy iPads becaue they are not allowed. Oh, and even books are banned too. Why? I've learnt there is rarely any point even asking, so I didn't bother.
So, I have about $4000 burning a hole in my pocket. We can't spend it on anything useful like books or paper charges, or anything fun like iPads. I guess that leaves boring and useless stuff. I have one last suggestion of an electric umbrella-dryer (don't laugh, we have seen only in the foyer of another lab, and can guess why it was bought...). Failing that, I could do what another now ex-colleague did a few years ago and buy another desktop computer and simply leave it in its box. In fact that is the machine I am currently using, as I snaffled it when they left last year (still unused).
Has anyone else got any other ideas?
Having only been in Japan during austere years of stagnation and budget cuts, I can hardly imagine what the bubble years must have been like...
[jules' pics] 8/13/2010 09:50:00 PM
As soon as we arrived James tried to blend in with some local culture. Unfortunately, despite his adopting the right attitude, people kept staring. Then we noticed, that although the streets were an array of diversity amazing to someone from the Kanto plain, inside the pub the dress code was blue jeans or else.
Thanks for complimenting London by thinking it was Paris! The other thing that is amazing to someone from the Kanto plain is the beautiful architecture, but I didn't have time to photograph any properly.
--
Posted By jules to jules' pics at 8/13/2010 09:50:00 PM
Friday, August 13, 2010
[jules' pics] 8/13/2010 05:37:00 AM
James in transit. There's a tandem called Garth in them there boxes and the double-reflection in the double-glazed windows is proof we are far from home.
I tend to recognise Japan-ness in the smallest details of photographs, so this must be a dead giveaway.
--
Posted By jules to jules' pics at 8/13/2010 05:37:00 AM
Thursday, August 12, 2010
I'm here!
As you might have guessed from jules' recent post, we have arrived. I see a lot of comments have accumulated over the last couple of days, and I'll be dealing with them when I have a bit more brain power. Which probably means over the weekend.
[jules' pics] 8/12/2010 05:29:00 AM
On the road again... but where to?
At time of ticket purchase, Virgin's Premium Economy beat standard Economy prices for the other airlines, so no illegal stress positions for us this time!
--
Posted By jules to jules' pics at 8/12/2010 05:29:00 AM
Tuesday, August 10, 2010
How not to compare models to data part eleventy-nine...
Not to beat the old dark smear in the road where the horse used to be, but...
A commenter pointed me towards this which has apparently been accepted for publication in ASL. It's the same sorry old tale of someone comparing an ensemble of models to data, but doing so by checking whether the observations match the ensemble mean.
Well, duh. Of course the obs don't match the ensemble mean. Even the models don't match the ensemble mean - and this difference will frequently be statistically significant (depending on how much data you use). Is anyone seriously going to argue on the basis of this that the models don't predict their own behaviour? If not, why on Earth should it be considered a meaningful test of how well the models simulate reality?
Of course the IPCC Experts did effectively endorse this type of analysis in their recent "expert guidance" note, where they remark (entirely uncritically) that statistical methods may assume that "each ensemble member is sampled from a distribution centered around the truth". But it's utterly bogus nevertheless, as there is no plausible situation in which that can occur, for any ensemble prediction system, ever.
Having said that, IMO a correct comparison of the models with these obs does show the consistency to be somewhat tenuous, as we demonstrated in that (in)famous Heartland presentation. It is quite possible that they will diverge more conclusively in the future. Or they may not. They haven't yet.
A commenter pointed me towards this which has apparently been accepted for publication in ASL. It's the same sorry old tale of someone comparing an ensemble of models to data, but doing so by checking whether the observations match the ensemble mean.
Well, duh. Of course the obs don't match the ensemble mean. Even the models don't match the ensemble mean - and this difference will frequently be statistically significant (depending on how much data you use). Is anyone seriously going to argue on the basis of this that the models don't predict their own behaviour? If not, why on Earth should it be considered a meaningful test of how well the models simulate reality?
Of course the IPCC Experts did effectively endorse this type of analysis in their recent "expert guidance" note, where they remark (entirely uncritically) that statistical methods may assume that "each ensemble member is sampled from a distribution centered around the truth". But it's utterly bogus nevertheless, as there is no plausible situation in which that can occur, for any ensemble prediction system, ever.
Having said that, IMO a correct comparison of the models with these obs does show the consistency to be somewhat tenuous, as we demonstrated in that (in)famous Heartland presentation. It is quite possible that they will diverge more conclusively in the future. Or they may not. They haven't yet.
[jules' pics] 8/09/2010 06:14:00 PM
The high alpine flowers on the rocky ridges tend to be of the blue-dot, white-dot or yellow-dot variety, which makes the orange lillies found in the lower flower meadows all the more striking.
The giant hogweed are pretty good too, but they were not yet out on our latest trip. Click here for a photo of that taken last year.
--
Posted By jules to jules' pics at 8/09/2010 06:14:00 PM
Monday, August 09, 2010
[jules' pics] 8/08/2010 08:56:00 PM
A country bumpkin from north west England, where it is very dark and matt, I considered the images of self that I found reflected on the city of London morally indecent, given their likelihood of encouraging vanity and obsession with appearance. It turns out that London has nothing on Japan where cleanliness is practised with great attention to detail, meaning that every surface that might possibly be reflective, is.
Just last week I read an ebook about photographing reflections, and realised that there was lots of reflective fun to be had, that entirely excluded vanity.
[Motomachi is a posh shopping street in Yokohama. Located in the foreign ghettos, it has quite a lot of western-style shops, although if you are over a size 12 UK woman you would have great trouble finding any clothes big enough]
--
Posted By jules to jules' pics at 8/08/2010 08:56:00 PM
Saturday, August 07, 2010
[jules' pics] 8/07/2010 01:56:00 AM
alpine flowers (anaphalis lactea f. alpicola) and the gendarmes, originally uploaded by julesberry2001.
3000m flowers are pretty small, and you have to grovel on the ground to take photos like this.
--
Posted By jules to jules' pics at 8/07/2010 01:56:00 AM
Friday, August 06, 2010
Wiley Interdisciplinary Reviews: Climate Change
A new journal has sprung up recently, I'm not entirely sure why or how, but it seems to be open access for now (not indefinitely) and has some interesting papers so maybe some of you would like to take a look. Called "Wiley Interdisciplinary Reviews: Climate Change" it seems to be a cross between an interdisciplinary journal and collection of encyclopaedic articles on climate change. There are a number of other WIREs journals on unrelated topics such as computational statistics, and nanomedicine and nanobiotechnology.
The Editors seem a bunch of slightly unconventional people, a little removed from the mainstream IPCC stalwarts though eminent enough and with some IPCC links: Hulme, Pielke, von Storch, Nicholls, Yohe are names that many will be familiar with. The others are probably all famous too, but I'm too ignorant to recognise them. I'm sure the journal is not intended as a direct rival to the IPCC, but it may turn out to provide an interesting and slightly alternative perspective.
The articles to date include a mix of authoritative reviews from leading experts - such as Parker on the urban heat island, Stott on detection and attribution, interspersed with perhaps more personal and less authoritative articles. I can safely say that without risk of criticism because one of them is mine - a review on Bayesian approaches to detection and attribution. This article had a rather difficult genesis. I was initially dubious about my suitability for the task and indeed the value of the article, but after declining once (and proposing another author, who also declined) I changed my mind and had a go. My basic difficulty with addressing the concept is that D&A has always seemed to me to be a rather limited and awkward approach to the question of estimating the effects of anthropogenic and natural forcing, which is tortured into a frequentist framework where it doesn't really fit. Eg, no sane person believes these forcings have zero effect, so what exactly is the purpose of a null hypothesis significance test in the first place? However, conventional D&A has such a stranglehold on the scientific conscious that most Bayesian approaches have actually mimicked this frequentist alternative of the Bayesian estimate that you really wanted in the first place. It all seems a bit tortured and long-winded to me.
Anyway, I eventually found some things to say, which hopefully aren't entirely stupid and help to show how a Bayesian approach might actually be useful in answering the questions that (sensible) people might want to know the answer to, rather than the relatively useless questions that frequentist methods can answer, which are then inevitably misinterpreted as answers to the questions that people wanted to answer in the first place (as I argue and document in the article).
Another of the personal and argumentative articles was contributed by Jules, who was invited to say something about skill and uncertainty in climate models. This was actually the article that sparked off our "Reliability" paper, as our discussions kept coming back to the odd inconsistency between the flat rank histogram evaluation that I know is standard in most ensemble prediction, versus the Pascal's triangle distribution that a truth-centred ensemble would generate (ie, if each model is independently and equiprobably greater than or less than the observations, then the obs should generally be very close to the ensemble median). Of course this problem didn't take long to solve once we had set out the issue clearly enough to recognise that there really were two incompatible paradigms in play, and Jules even ended up citing the GRL paper which overtook her WIREs one in the review process.
Perhaps of more widespread interest to other readers, is a simple analysis of the skill of Hansen's forecast which he made back in 1988 to the US Congress. We'd actually had lengthy discussions with several people (listed in the acknowledgements) a year or two ago, trying to resurrect the old model code that was used for this prediction in order to re-run it and analyse its outputs in more detail. But this proved to be impossible. (The code exists but has been updated and gives substantially different results. If only the code had been published in GMD!) Therefore we were left with nothing more than the single printed plot of global mean temperature to look at. This didn't seem much to base a proper peer-reviewed paper on, so the idea died a death. When this WIREs invitation came long it seemed like a good opportunity to publish the one usable result we had obtained, as an example of what skill means. The headline result is that under any reasonable definition of skill, the Hansen prediction was skillful. While no great surprise, I don't think it has been presented in quite those terms before. It's a shame that we weren't able to generate a more comprehensive set of outputs though which might have given a more robust result than this single statistic.
The null hypothesis of persistence (no change in temperature) was found to give best performance over the historical interval, compared to extrapolating a trend. So this is the appropriate zero-skill baseline for evaluating the forecast. Nowadays with the AGW trend well established, probably most would argue that a continuation of that trend is a good bet, though that still leaves open the question of how long a historical interval to fit the trend over. Anyway, the model forecast is clearly drifting on the high side by now - most likely due to some combination of high sensitivity, low thermal inertia and lack of tropospheric aerosols - but is still far closer to the observations than we would have achieved by assuming no change. Furthermore, the observed warming is also very close to the top end of the distribution of historical 20 year trends, meaning that the observed outcome would be very unlikely if the the climate was merely following some sort of random walk. This evidence for the power of climate models is obviously limited by lack of detailed outputs for validation, but what there is is clearly very strongly supportive.
The Editors seem a bunch of slightly unconventional people, a little removed from the mainstream IPCC stalwarts though eminent enough and with some IPCC links: Hulme, Pielke, von Storch, Nicholls, Yohe are names that many will be familiar with. The others are probably all famous too, but I'm too ignorant to recognise them. I'm sure the journal is not intended as a direct rival to the IPCC, but it may turn out to provide an interesting and slightly alternative perspective.
The articles to date include a mix of authoritative reviews from leading experts - such as Parker on the urban heat island, Stott on detection and attribution, interspersed with perhaps more personal and less authoritative articles. I can safely say that without risk of criticism because one of them is mine - a review on Bayesian approaches to detection and attribution. This article had a rather difficult genesis. I was initially dubious about my suitability for the task and indeed the value of the article, but after declining once (and proposing another author, who also declined) I changed my mind and had a go. My basic difficulty with addressing the concept is that D&A has always seemed to me to be a rather limited and awkward approach to the question of estimating the effects of anthropogenic and natural forcing, which is tortured into a frequentist framework where it doesn't really fit. Eg, no sane person believes these forcings have zero effect, so what exactly is the purpose of a null hypothesis significance test in the first place? However, conventional D&A has such a stranglehold on the scientific conscious that most Bayesian approaches have actually mimicked this frequentist alternative of the Bayesian estimate that you really wanted in the first place. It all seems a bit tortured and long-winded to me.
Anyway, I eventually found some things to say, which hopefully aren't entirely stupid and help to show how a Bayesian approach might actually be useful in answering the questions that (sensible) people might want to know the answer to, rather than the relatively useless questions that frequentist methods can answer, which are then inevitably misinterpreted as answers to the questions that people wanted to answer in the first place (as I argue and document in the article).
Another of the personal and argumentative articles was contributed by Jules, who was invited to say something about skill and uncertainty in climate models. This was actually the article that sparked off our "Reliability" paper, as our discussions kept coming back to the odd inconsistency between the flat rank histogram evaluation that I know is standard in most ensemble prediction, versus the Pascal's triangle distribution that a truth-centred ensemble would generate (ie, if each model is independently and equiprobably greater than or less than the observations, then the obs should generally be very close to the ensemble median). Of course this problem didn't take long to solve once we had set out the issue clearly enough to recognise that there really were two incompatible paradigms in play, and Jules even ended up citing the GRL paper which overtook her WIREs one in the review process.
Perhaps of more widespread interest to other readers, is a simple analysis of the skill of Hansen's forecast which he made back in 1988 to the US Congress. We'd actually had lengthy discussions with several people (listed in the acknowledgements) a year or two ago, trying to resurrect the old model code that was used for this prediction in order to re-run it and analyse its outputs in more detail. But this proved to be impossible. (The code exists but has been updated and gives substantially different results. If only the code had been published in GMD!) Therefore we were left with nothing more than the single printed plot of global mean temperature to look at. This didn't seem much to base a proper peer-reviewed paper on, so the idea died a death. When this WIREs invitation came long it seemed like a good opportunity to publish the one usable result we had obtained, as an example of what skill means. The headline result is that under any reasonable definition of skill, the Hansen prediction was skillful. While no great surprise, I don't think it has been presented in quite those terms before. It's a shame that we weren't able to generate a more comprehensive set of outputs though which might have given a more robust result than this single statistic.
The null hypothesis of persistence (no change in temperature) was found to give best performance over the historical interval, compared to extrapolating a trend. So this is the appropriate zero-skill baseline for evaluating the forecast. Nowadays with the AGW trend well established, probably most would argue that a continuation of that trend is a good bet, though that still leaves open the question of how long a historical interval to fit the trend over. Anyway, the model forecast is clearly drifting on the high side by now - most likely due to some combination of high sensitivity, low thermal inertia and lack of tropospheric aerosols - but is still far closer to the observations than we would have achieved by assuming no change. Furthermore, the observed warming is also very close to the top end of the distribution of historical 20 year trends, meaning that the observed outcome would be very unlikely if the the climate was merely following some sort of random walk. This evidence for the power of climate models is obviously limited by lack of detailed outputs for validation, but what there is is clearly very strongly supportive.[jules' pics] 8/06/2010 12:55:00 AM
At this time of year the Japanese Alps are covered in flowers. The flowers get smaller as you get higher. This one is from around 2600m.
P.S. The flower is a Pedicularis chamissonis var.Japonica--
Posted By jules to jules' pics at 8/06/2010 12:55:00 AM
Thursday, August 05, 2010
"IPCC Experts" New Clothes
You may recall not so long I ago I blogged about our paper in which we argued that the (standard outside climate science) paradigm of a statistically indistinguishable ensemble - where reality is just another sample from the distribution - is a much more natural and plausible interpretation of the AR4 multi-model ensemble, than the alternative "truth-centred" paradigm - where the models are assumed to be scattered around with reality lying exactly at the centre of their sampling distribution. The latter has no theoretical basis or practical support as far as I can tell, it appears to have been plucked out of thin air by a process of wishful thinking, and is strongly refuted by an analysis of the ensemble. But this post isn't really about that.
Immediately after that paper was published, the IPCC held a closed meeting which we were of course not permitted to attend. The purpose of the meeting was to generate a "best practice guidance paper" for the use of the multi-model ensemble. Jules predicted that our work would get misinterpreted somehow, but I thought our paper was fairly straightforward and hard to misunderstand. Well, I hadn't reckoned on the unique skills of the "IPCC Experts". Eventually this meeting report and summary appeared on their web site.
Regarding the interpretation of the multi-model ensemble, they say:
What? What is "the space of all possible but equally credible climate models" and what does this have to do with anything? Of the papers they cite, only ours actually mentions exchangeability and statistical indistinguishability, and what we wrote is that this means that "the truth is drawn from the same distribution as the ensemble members, and thus no statistical test can reliably distinguish one from the other". We also cited Toth et al 2003 (good book by famous NWP people) who wrote equivalently "the ensemble members and the verifying observation are mutually independent realizations of the same probability distribution".
Note that there is no reference to the "space of all possible models". All that matters is that the sampling distributions of models and truth are the same.
This may appear at first to be a rather pedantic and minor complaint. However, it doesn't take long to realise that the "space of all possible models" is a "colourless green idea", that is, a syntactically valid but completely meaningless phrase. This isn't just my assertion, it is agreed by all the previous authors who have used this terminology! (If you wish to disagree, feel free to explain in the comments what a "possible model" is, and how it can be distinguished from an impossible one. Good luck with that.)
In fact as far as we can tell this phrase has only ever been used to denigrate the use of the multi-model ensemble. The argument goes, that in order to understand how to use this ensemble, we have to first understand the "space of all possible models" from which they are sampled. This phrase is meaningless, therefore the use of the ensemble is theoretically ill-founded. Supporting quotes are appended below - quotes which many attendees of the meeting were well aware of, because they wrote them. Well, we don't mind people writing gibberish in their own papers, but we object strongly to them linking such nonsense to our work. Our analysis does not depend in any way on this meaningless concept, and to claim that it does (with the corollary that our analysis is philosophically ill-founded) is a flat-out lie.
Immediately after that paper was published, the IPCC held a closed meeting which we were of course not permitted to attend. The purpose of the meeting was to generate a "best practice guidance paper" for the use of the multi-model ensemble. Jules predicted that our work would get misinterpreted somehow, but I thought our paper was fairly straightforward and hard to misunderstand. Well, I hadn't reckoned on the unique skills of the "IPCC Experts". Eventually this meeting report and summary appeared on their web site.
Regarding the interpretation of the multi-model ensemble, they say:
Alternatively, a method may assume:
b. that each of the members is considered to be ‘exchangeable’ with the other members and with the real system (e.g., Murphy et al., 2007; Perkins et al., 2007; Jackson et al., 2008; Annan and Hargreaves, 2010). In this case, observations are viewed as a single random draw from an imagined distribution of the space of all possible but equally credible climate models and all possible outcomes of Earth’s chaotic processes.
What? What is "the space of all possible but equally credible climate models" and what does this have to do with anything? Of the papers they cite, only ours actually mentions exchangeability and statistical indistinguishability, and what we wrote is that this means that "the truth is drawn from the same distribution as the ensemble members, and thus no statistical test can reliably distinguish one from the other". We also cited Toth et al 2003 (good book by famous NWP people) who wrote equivalently "the ensemble members and the verifying observation are mutually independent realizations of the same probability distribution".
Note that there is no reference to the "space of all possible models". All that matters is that the sampling distributions of models and truth are the same.
This may appear at first to be a rather pedantic and minor complaint. However, it doesn't take long to realise that the "space of all possible models" is a "colourless green idea", that is, a syntactically valid but completely meaningless phrase. This isn't just my assertion, it is agreed by all the previous authors who have used this terminology! (If you wish to disagree, feel free to explain in the comments what a "possible model" is, and how it can be distinguished from an impossible one. Good luck with that.)
In fact as far as we can tell this phrase has only ever been used to denigrate the use of the multi-model ensemble. The argument goes, that in order to understand how to use this ensemble, we have to first understand the "space of all possible models" from which they are sampled. This phrase is meaningless, therefore the use of the ensemble is theoretically ill-founded. Supporting quotes are appended below - quotes which many attendees of the meeting were well aware of, because they wrote them. Well, we don't mind people writing gibberish in their own papers, but we object strongly to them linking such nonsense to our work. Our analysis does not depend in any way on this meaningless concept, and to claim that it does (with the corollary that our analysis is philosophically ill-founded) is a flat-out lie.
In fact the multi-model ensemble can be very naturally interpreted as sampling our collective uncertainties about how best to represent the climate system. The question of reliability of the ensemble then simply amounts to asking whether these uncertainties are well-calibrated or not - which as we have shown, is an eminently testable hypothesis (at least in respect of current and historical data) and does not require anyone to "imagine" such bizarre and spurious constructions as the "space of all possible models".
We complained to the authors of this piece of nonsense, and they replied with the remarkable claim that despite being listed as the authors, they were not in fact responsible for the accuracy of anything they wrote, as they were merely reporting the "the definition as determined and agreed by the attendees", and would not countenance any correction of this mistake. Yes, they really used those words I have placed in quotes. Apparently it didn't occur to any of these "experts" present that this concept of statistical indistinguishability was an established term of art that already had a perfectly adequate definition, and that this existing definition is the only one that has ever been presented in the context of climate science. Their decision to reinvent the definition of statistical indistinguishability apparently has the full support of the IPCC hierarchy. I'm utterly gobsmacked that they place their duty to defend this "consensus" of a private clique above their duty to ensure that this "consensus" is honest, accurate, and useful to potential readers, let alone providing a fair representation of the work of those who are prohibited from participation in this process. It's as if the WG2 authors had simply proclaimed that 2035 was the date the experts had agreed that all Himalayan glaciers would vanish, and that was the end of the matter.
We have various manuscripts at different stages of writing and review, and can probably correct this mistake somehow (assuming that reviewers allow us to dissent from the newly-established "consensus"), but it's unlikely that what we write will ever have the circulation and influence that the IPCC bully pulpit affords. And of course, it is pretty hard to proof our work against spurious criticism when these "experts" are prepared to simply pluck arbitrary nonsense out of thin air. It's a shame that no-one there actually stood up and said "But these words have no meaning, how can they be used in a definition?"
Some references to the "space of all possible models", which make the nonsensical nature of this phrase clear, and how it has been used to argue against the use of the multi-model ensemble:
Allen et al 2002:
"the distribution of all possible models is undefined"
Collins 2007:
"Is the collection of the world’s climate models an adequate sample of the space of all possible models (and, indeed, is it even possible to define such a space)?"
Murphy et al 2007:
"Specifically, it is not clear how to define a space of possible model configurations of which the MME members are a sample. This creates the need to make substantial assumptions in order to obtain probabilistic predictions from their results"
Stainforth et al 2007:
"The lack of any ability to produce useful model weights, and to even define the space of possible models, rules out the possibility of producing meaningful PDFs for future climate based simply on combining the results from multi-model or perturbed physics ensembles; or emulators thereof."
We complained to the authors of this piece of nonsense, and they replied with the remarkable claim that despite being listed as the authors, they were not in fact responsible for the accuracy of anything they wrote, as they were merely reporting the "the definition as determined and agreed by the attendees", and would not countenance any correction of this mistake. Yes, they really used those words I have placed in quotes. Apparently it didn't occur to any of these "experts" present that this concept of statistical indistinguishability was an established term of art that already had a perfectly adequate definition, and that this existing definition is the only one that has ever been presented in the context of climate science. Their decision to reinvent the definition of statistical indistinguishability apparently has the full support of the IPCC hierarchy. I'm utterly gobsmacked that they place their duty to defend this "consensus" of a private clique above their duty to ensure that this "consensus" is honest, accurate, and useful to potential readers, let alone providing a fair representation of the work of those who are prohibited from participation in this process. It's as if the WG2 authors had simply proclaimed that 2035 was the date the experts had agreed that all Himalayan glaciers would vanish, and that was the end of the matter.
We have various manuscripts at different stages of writing and review, and can probably correct this mistake somehow (assuming that reviewers allow us to dissent from the newly-established "consensus"), but it's unlikely that what we write will ever have the circulation and influence that the IPCC bully pulpit affords. And of course, it is pretty hard to proof our work against spurious criticism when these "experts" are prepared to simply pluck arbitrary nonsense out of thin air. It's a shame that no-one there actually stood up and said "But these words have no meaning, how can they be used in a definition?"
Some references to the "space of all possible models", which make the nonsensical nature of this phrase clear, and how it has been used to argue against the use of the multi-model ensemble:
Allen et al 2002:
"the distribution of all possible models is undefined"
Collins 2007:
"Is the collection of the world’s climate models an adequate sample of the space of all possible models (and, indeed, is it even possible to define such a space)?"
Murphy et al 2007:
"Specifically, it is not clear how to define a space of possible model configurations of which the MME members are a sample. This creates the need to make substantial assumptions in order to obtain probabilistic predictions from their results"
Stainforth et al 2007:
"The lack of any ability to produce useful model weights, and to even define the space of possible models, rules out the possibility of producing meaningful PDFs for future climate based simply on combining the results from multi-model or perturbed physics ensembles; or emulators thereof."
Wednesday, August 04, 2010
[jules' pics] 8/03/2010 08:53:00 PM
After a relatively lonely walk for several days, when we came to the Yari ridge it became a game of dodging the hoards of grockles. This turned out surprisingly well thanks to James' superior planning, and only our last night was spent at a busy hut. Unusually we were also not the only gaijin in the village.To prove it, here is one, just about to dive off down the Daikiretto.
--
Posted By jules to jules' pics at 8/03/2010 08:53:00 PM
Tuesday, August 03, 2010
Not so hot
Another month of HadCRU data came in some time ago, so I can update this previous post. It's still not looking too good for me.
First ENSO continues to fade away, with a La Nina forecast for the summer, and predicted to be a bit stronger than the equivalent in 1998 (probably anything up to about Sept may affect the 2010 temperature):
And now the temperatures, which are actually still rising in terms of 12 month lagged average, but which have now dropped marginally below the 1998 values. There is not a lot in it yet, but 1998 had two more really hot months still to come at this point, and as you can see from the solid lines almost every month in 1998 was warmer than its equivalent in 2010:
The year to date average from HadCRU is also just a whisker below the final 1998 value (0.520 vs 0.529).
First ENSO continues to fade away, with a La Nina forecast for the summer, and predicted to be a bit stronger than the equivalent in 1998 (probably anything up to about Sept may affect the 2010 temperature):
And now the temperatures, which are actually still rising in terms of 12 month lagged average, but which have now dropped marginally below the 1998 values. There is not a lot in it yet, but 1998 had two more really hot months still to come at this point, and as you can see from the solid lines almost every month in 1998 was warmer than its equivalent in 2010:
The year to date average from HadCRU is also just a whisker below the final 1998 value (0.520 vs 0.529).
[jules' pics] 8/02/2010 09:04:00 PM
There is an infamous deep cut (daikiretto) towards the left of the famous Kita Alps ridge. It is about 300m deep, and is quite a narrow ridge with big drops all around. We have done it before but this time we did it the easier way from North to South (right to left on yesterday's panorama). Having been fortunate enough to survive the first time, I was a bit apprehensive about doing it again. Turned out it was really easy. Does that look scary to you? It is true that if you jumped off hard enough, you'd be falling for 100s of meters, but why would you do that?
Click here for an alternative take. It wasn't icy when we did it, of course, which would make it much more difficult. On an earlier trip this year we viewed the daikiretto from the south (north end of it visible here), and the path in the snow squiggling down the ridge was quite worrying.
--
Posted By jules to jules' pics at 8/02/2010 09:04:00 PM
Monday, August 02, 2010
[jules' pics] 8/01/2010 08:32:00 PM
The sun strikes the most famous Kita Alps ridge at just before raw egg o'clock.
The original, 14148 pixels across, is a panorama made up from about 7 photos from James' LX3. I think that interwebs do it no justice and intend to print it out several feet across and stick to to a wall somewhere. You can too! Here it is on SmugMug (split in two for reasons of economy).
--
Posted By jules to jules' pics at 8/01/2010 08:32:00 PM
Sunday, August 01, 2010
[jules' pics] 8/01/2010 03:05:00 AM
Good eating on a raicho (basically a ptarmigan) I reckon, but unfortunately it too is off the menu in Japan, due to being a cute and treasured symbol of the mountains. So whenever we see one we exact revenge on any Japanese within earshot by exclaiming the same phrase they use when looking at horrible sea monsters at the aquarium .. "oishiiiii sou"... (It looks delicious).
Enjoy the grouse season everyone...
--
Posted By jules to jules' pics at 8/01/2010 03:05:00 AM
Subscribe to:
Posts (Atom)