Saturday, June 17, 2006

Two strikes...

And this time the Comment on Hegerl et al is definitely in the bin. No surprise there, and I did appreciate getting a more reasonably-worded explanation along with the rejection this time. It still seems odd that they say they might be prepared to consider it if and only if we remove any suggestion that it has implications beyond that one specific paper, but they have obviously made their final decision on that point. I'm not going to pursue that approach for the time being - we are off to the UK next week for a workshop on uncertainty in climate change, and I intend to talk about these issues so hopefully we'll get some useful feedback from a wider audience.

Meanwhile, the Comment on Frame et al has been revised and resubmitted, and can be found here. In light of the reply and reviews, we've tried to point out more explicitly where we have significant points of disagreement. We'll probably see how that fares before planning our next move. In the time it's taken, we could probably have got a stand-alone paper done and dusted, but I still think there are good reasons for pursuing it as a Comment and Reply.

4 comments:

Anonymous said...

I probably ought to read Rougier 2006 and Bernardo and Smith Chapter 5.4 and 5.6 before posting this but.....

Apart from being a huge amount of work, what do you think of the following approach?

Take one prior and use it to get an posterior pdf. Then use that pdf as the prior. Repeat this a few times to see where it settles/oscilates.

Now start again with a different prior and repeat the process.

If the two answers are close, you (might?) have what could be considered an objective pdf. If they are not close you have some sort of measure of the extent to which the prior matters.

(For the 2 original priors you could ask 20 experts to provide a suitable prior and use the mean plus one standard deviation and the mean minus one standard deviation.)

crandles

Anonymous said...

I am obviously stupid, I missed out a crucial stage. There is a need to spread out the resulting pdf shape to remove the knowledge gained in the process. Is keeping the shape and just increasing the standard deviation possible or will this introduce a bias into the final shape?

I am basically just aiming for a stable shape. (Myles Allen, How can we (in)validate a probabilistic forecast? Aim for a STAID forecast http://www.climateprediction.net/science/pubs/ecmwf02.pdf) Trying to think about this method (probably with little success), I think it could easily give too much weight to new observations.

crandles

James Annan said...

I think we've already established that the prior doesn't matter much unless we (a) use an extraordinary one and (b) ignore almost all the evidence that we have.

All that remains is to convince everyone else about it :-)

Brian said...

James, somewhat off-topic, but your bet with the Russians was referenced during a popular radio science show, Science Friday. More info about the show here:

http://scienceblogs.com/intersection/2006/06/science_friday_podcast_boingbo.php

I'd guess it came up about a half-hour into the show, just briefly. Thought I'd let you know if you hadn't heard.