SteveF pointed me to a comment and reply in Nature this week on the Hegerl et al paper.
To start with, here's a brief disclaimer - I'm blogging this "blind", having had a very brief glance at the letters at work last week but with no access to them here at home over the weekend. With that in mind...
Schneider (no, not that one) complains that Hegerl et al haven't fully accounted for some uncertainty in their estimate. I haven't looked carefully enough to check if he is technically correct in principle, but even if he is, that wouldn't necessarily mean that their reply ("the method...does account for uncertainty in reconstructions") is wrong. Researchers are entitled to make what they consider to be reasonable approximations. Ideally, they would discuss and justify these in detail but in reality that's not going to happen in a 4-page letter. Tellingly, Schneider does not estimate the effect of this supposed error on their results, but only provides a example of how it could potentially matter in a pathological case. I score it as a win for Hegerl et al.
In closing their reply, Hegerl et al mention that they could have used other lines of evidence to tighten their result (citing us and Thomas Schneider von Deimling's paper), thus they consider it to be conservative and valid. I'll not bother to quibble about the apparent incompatibility of these two terms (even assuming I've remembered them correctly), but it is disappointing that they didn't find the space to also observe that if they hadn't started out from the extremely alarmist prior belief that eg P(S>6C)=40%, then they would have generated a much lower posterior probability of this event anyway. But what's much worse than that IMO is the fact that Nature made the decision to publish discussion of what amounts to a technical detail that has no demonstrable impact on the result, but would not consider our point (which has a substantial practical impact) that the basic approach of this and much other similar work is fundamentally wrong.
It will certainly be interesting to see how this all plays out over the next year or two.
To start with, here's a brief disclaimer - I'm blogging this "blind", having had a very brief glance at the letters at work last week but with no access to them here at home over the weekend. With that in mind...
Schneider (no, not that one) complains that Hegerl et al haven't fully accounted for some uncertainty in their estimate. I haven't looked carefully enough to check if he is technically correct in principle, but even if he is, that wouldn't necessarily mean that their reply ("the method...does account for uncertainty in reconstructions") is wrong. Researchers are entitled to make what they consider to be reasonable approximations. Ideally, they would discuss and justify these in detail but in reality that's not going to happen in a 4-page letter. Tellingly, Schneider does not estimate the effect of this supposed error on their results, but only provides a example of how it could potentially matter in a pathological case. I score it as a win for Hegerl et al.
In closing their reply, Hegerl et al mention that they could have used other lines of evidence to tighten their result (citing us and Thomas Schneider von Deimling's paper), thus they consider it to be conservative and valid. I'll not bother to quibble about the apparent incompatibility of these two terms (even assuming I've remembered them correctly), but it is disappointing that they didn't find the space to also observe that if they hadn't started out from the extremely alarmist prior belief that eg P(S>6C)=40%, then they would have generated a much lower posterior probability of this event anyway. But what's much worse than that IMO is the fact that Nature made the decision to publish discussion of what amounts to a technical detail that has no demonstrable impact on the result, but would not consider our point (which has a substantial practical impact) that the basic approach of this and much other similar work is fundamentally wrong.
It will certainly be interesting to see how this all plays out over the next year or two.
2 comments:
So are you saying this sort of thing is bias in a journal? Potentially increased uncertainty is newsworthy so it gets published but if you say there is less uncertainty it isn't newsworthy and doesn't get published despite being a more important consideration.
BTW I guess the bet being discussed on pages 57 and 58 see
http://www.climateprediction.net/board/viewtopic.php?t=3004&postdays=0&postorder=asc&start=840
is going nowhere with him having offered the bet then hiding behind the difficulty of judging it correctly. Still that might be more clear if I have suggested IPCC 5AR and I also suggest some sort of expert panel. How much would you charge to judge such a claim? Any other thoughts on how that bet could be judged or more generally about that bet?
crandles
Chris,
I wouldn't like to say what the "bias" is exactly - the Hegerl et al paper itself was touted as reducing the uncertainty, so I'm sure that further progress in that direction will be loudly trumpeted when the right person says it. But whatever the reasoning behind it, the main outcome of the editors' decisions has been to obstruct what is obviously a very valid and important scientific debate - and that's assuming one accepts that the Hegerl/Frame/Allen POV is remotely valid in the first place :-)
Regarding your "bet", it sounds like typical faith-based bluster of a denialist. He needs to work out what would quantifiably support or refute his POV - if there is nothing, it's not a scientific proposition.
Post a Comment