Another week, another rejection (without review) from Nature. This was an attempted comment on Hegerl et al, which can be found here. The points we were trying to make will not come as a surprise to anyone who has read my previous comments such as the comment on Frame et al which we are still waiting to hear anything about, after a full 2 months at GRL. Basically, there are two main reasons why Hegerl et al's "pdf" is not actually a valid probabilistic estimate of climate sensitivity at all. Firstly, they ignore much of the data that bears on the matter (and which indicates a highest likelihood of a value of about 3C), and secondly by starting off with a prior that assigns very high probability to high sensitivity and ignoring most of the evidence to the contrary, they ensure that the result also has a high probability of high sensitivity - albeit far lower than their prior did. Of course these...limitations...are prevalent in much of the literature.
Nature's excuse this time? Editor Nicki Stevens wrote:
Meanwhile, we have people like Gavin Schmidt quite prepared to openly dismiss the bulk of peer-reviewed literature in this area with such comments as "Basically no one really believes that those really high sensitivities are possible," and "even Hegerl's top limit is too high". Not that I'm criticising him for that - quite the reverse, but the fact that there is such a credibility gap between what has appeared in the literature, and what at least some responsible and reputable scientists think, should surely be seen as rather worrying by all who are interested in ensuring that the scientific process works as intended. It is quite clear that (unless our arguments are wholly invalid, and so far no-one has suggested why they should be) none of the published "pdfs" actually provide any credible support for the belief that S is greater than 6C even at as little as the 5% level (for example), but according to Nature, as long as everyone keeps on getting this wrong together, they aren't interested in correcting the mistaken (and alarmist) impression that they have helped to foster. We feel like the boy who tells the emperor that he has no clothes, except that we are not even being allowed to say it, at least not anywhere that it will be seen.
Nature's excuse this time? Editor Nicki Stevens wrote:
we have regretfully decided that publication of this comment as a Brief Communication Arising is not justified, as the concerns you have raised apply more generally to a widespread methodological approach, and not solely to the Hegerl et al. paperYes, you read that right. Because everyone else has been doing much the same thing, they aren't interested in ensuring that the stuff they publish is valid. Really, there seems little answer to this beyond picking our jaws off the floor and keeping it in mind when we read future Nature papers. (FWIW, it was also Nicki Stevens who told us that our GRL manuscript didn't provide enough of an "advance in significantly constraining climate sensitivity relative to prior estimates".)
Meanwhile, we have people like Gavin Schmidt quite prepared to openly dismiss the bulk of peer-reviewed literature in this area with such comments as "Basically no one really believes that those really high sensitivities are possible," and "even Hegerl's top limit is too high". Not that I'm criticising him for that - quite the reverse, but the fact that there is such a credibility gap between what has appeared in the literature, and what at least some responsible and reputable scientists think, should surely be seen as rather worrying by all who are interested in ensuring that the scientific process works as intended. It is quite clear that (unless our arguments are wholly invalid, and so far no-one has suggested why they should be) none of the published "pdfs" actually provide any credible support for the belief that S is greater than 6C even at as little as the 5% level (for example), but according to Nature, as long as everyone keeps on getting this wrong together, they aren't interested in correcting the mistaken (and alarmist) impression that they have helped to foster. We feel like the boy who tells the emperor that he has no clothes, except that we are not even being allowed to say it, at least not anywhere that it will be seen.
36 comments:
Oh dear. Gavin thinks your results are barely of interest because everyone knows this anyway; Nature won't publish because you're too far away from what they've published before...
Amusingly, I had this same thing from 2 reviewers on a paper on sea ice dynamics; happily in that case I was able to convince the editor to re-think.
The only surprise, James, is that you're even surprised by Nature's lackadaisical approach. You'd expect that post-Hwang woo Suk, they'd be much more careful, and much more accommodating of critiques - and you;d be disappointed.
It's clear to me at least, that Nature has an agenda in the same way that Fox News has an agenda. Doubt of any kind about alarmist predictions (and Gavin Schmidt should know about this) is ruthlessly censored. It doesn't fit the script of the agenda.
One of the reasons that McIntyre and McKitrick were not published by Nature was that it was "too complex" for Nature's readership.
Now you may take the view that climate skeptics are wrong about climate science, but funnily enough you're now seeing how "scientific consensuses" are created and maintained - and its not necessarily through force of argument.
"Everyone knows this anyway" is one possible point of view, but in that case why have such views not appeared in the peer-reviewed literature? It is IMO wishful thinking to imagine that the reason why no-one has said climate sensitivity isn't really as high as 6C, is because this statement is too obvious to be worth making! There are clearly some people who think that it has been "proved that climate sensitivity has a long tail" (to high sensitivity) or similar nonsensical wording.
According to GRL, we should shortly get a reply to the GRL comment, at which point we will at least see whether Frame et al think that our comments are obviously correct.
As for Jonh A's comments - I am under no illusion that scientists are people too. Nevertheless, I am also sure that our view is gaining traction among researchers in the field, and will continue to do so. The main question is simply how long it will take. Nature hinders things by putting a largely ignorant editor in the way, but the GRL paper got published and there may be others to follow if people continue to make misleading (IMO) statements.
1. I think you are better off in a serious specialty journal than in Nature for articles on methodology. Even though methodology is vitally important in moving science forward, it is not what Nature specializes in. Plus you will reach more of who should care and who should criticize you in the specialty journal. Not sure if a "letters" journal is desirable either. But GRL better than Nature.
2. Have you seen the criticism by SM of Hegerl's confidance intervals? I think his bigger criticisms relate to the proxy study itself (CH-blend). But the confidance interval thing just seems like a clear math error. Nothing over damning in that by the way...but Gabi needs to set the record right immediately. Embarressing that no reply made to the notes on the seemingly clear error.
TCO,
1. You are probably right, although some might say the "methodology" is too trivial to be worth publishing in its own right (and we've already mostly done it with the GRL paper). Anyway, if we are going to specifically criticise other work, it seems only fair to give the original authors the right of reply (which a Comment/Reply does).
2. I presume you mean this? Sure, it looks a bit embarassing (even for Dave Frame, I would have thought). Of course, we all make mistakes...
Of course, we all make mistakes...
...and if lots of people make the same mistakes, Nature doesn't even bother to publish corrections.
Which gets me back to what good is peer review? If your peers know less about mathematics and logic than you do, how does quality science result? Is it like magic?
how does quality science result? Is it like magic?
Basically, yes it is :-) After all, your computer works, planes fly, medicines are effective - despite the fallibility of the people who research such things. It's clear that people have a tendency to hold on to beliefs for some time after they cease to be rationally tenable, and it's frustrating to be on the wrong end of this. But ultimately the truth does generally come out - and "ultimately" isn't necessarily a long time in the grand scheme of things.
Peer review is what stops journals looking like usenet, or even blogs :-) No-one claims it is perfect, but no-one has found a clearly better alternative (although there are some ongoing experiments). IMO the problems at Nature have more to do with it's editorial focus on "science news" first, rather than science.
I don't want to get into some simpleton overdwelling on an error (or to impute that one error is a sign of further or of stupidity)*, but still...that upper conf limit crossing the lower is fricking cool! It makes my drunken heart happy. Sing a song of joy!
*But wouldn't rule it out...
my authentication: foongnm! (say it with pride)
Oh...and react a little more to my arch putdown of letters journals. (I think applied physics letters ought to be renamed applied marketing letters...and that it is about 3 steps below any chem or mat sci journal...so mich for those smarty 'zoics types.)
Oh...and methodology IS IMPORTANT. It is vital. You don't even need to argue it. it's foundational. Doing work that gets into a textbook, should be every scientist's heart desire.
Not as snappy: csevhjgx
The letters have their place, but generally need to be accompanied by more detailed papers (although perhaps "supplementary info" can fill a similar role). Hmm...I guess I am at risk of talking myself into a full-length version of these recent comments.
I want to try to avoid sounding too bitter and twisted, but certainly the Nature letters are often little more than some sort of extended abstract/press release. Even GRL allows a whole lot more space.
My comments about this funny story are here.
Lumo,
Keep taking the tablets.
I want to try to avoid sounding too bitter and twisted, but certainly the Nature letters are often little more than some sort of extended abstract/press release. Even GRL allows a whole lot more space.
You better be careful that you don't get labelled a "skeptic" for wanting the very same scientific transparency that Steve McIntyre has championed.
The fact that Nature has such an "editorial focus" which extends to blocking criticism of clearly false or exaggerated results ought to tell you something important.
Peter,
It's always possible that I'm wrong :-) But note that Nature's editorial pre-review judgements are explicitly made on grounds other than wrongness or rightness.
I suspect that Nature just decides what are the top priorities for newsworthiness, and then makes up some boilerplate excuse to bin the rest. I hope their decisions are due to their viewing the world through the eyes of a journalist, rather than any deep conspiracy. But when the excuse doesn't even pass the sniff test (like on my last two attempts) it is rather frustrating.
The solution (John A and other sceptics take note) is to plug away at presenting a coherent case elsewhere in the peer-reviewed literature, rather than just ranting about how unfair the world is. But there's no harm in doing a little of the latter, and a blog is as good a place as any for it :-)
"Is it possible..." -PH
Damn, dude. READ the papers. Read the blog articles. And THINK. Sheesh. You seem to show no independant thinking ability.
The solution (John A and other sceptics take note) is to plug away at presenting a coherent case elsewhere in the peer-reviewed literature, rather than just ranting about how unfair the world is. But there's no harm in doing a little of the latter, and a blog is as good a place as any for it :-)
I've not given up on peer-reviewed literature. I have given up on formerly respectable science periodicals which appear to have been captured by an exclusivist clique.
I am myself composing a scientific article (fortunately not on climate science) for prospective publication in a peer-reviewed journal. I think the catch is, that I will be going for a specialist journal where my peers will probably know more than I do.
I notice that Peter Hearnden is here to do the bidding of the "consensus". As you'll find out, his view of science is very post-modern. If it's in line with his personal beliefs, its a fact beyond dispute. If not, then it's merely your "opinion"
I think scientists should be challenged to defend their work. I think your criticism, right or wrong, (and I personally think it has a lot of merit) should be published. The excuse given by Nature for not doing so is the latest in a series of pathetic excuses that ultimately undermine the scientific method.
Dear James
thanks for the amusing anecdote.
I have to say that your "rant" starts to sound suspiciously like scepticism; but that couldn't possibly be :)
yours
per
Dear per,
the word skepticism is probably too strong. The story reminds me of the different subgroups of a certain political party in a certain bloc in the 1950s who were executing each other. ;-)
James: I am not a Tablet PC user.
Best
Lubos
There is quite a gulf between the rational scepticism that I hope all scientists have, and the silly denial of some on the extreme. I've always considered myself firmly in the first camp.
If you read what I've had to say about (eg) Lovelock, Bryden, and some of the climate sensitivity work, you'll see that this latest rant is hardly a radical departure - and note that the realclimate contributors have posted somewhat similar comments in many cases too.
And as an afterthought on s(c)epticism, I am not unduly worried by the occasional link from places like TCS and climateaudit, but if the oil and coal industry start to offer me money, I'll know I've gone too far :-)
Science, Nature and Physical Review Letters are particular problems. For most journals the editor will always send your paper out for review. In those three cases (know of others?), the editors first make a judgement about whether the article should be reviewed.
That tends to be a very "our gang" judgement. There is not very much you can do about it.
And as an afterthought on s(c)epticism, I am not unduly worried by the occasional link from places like TCS and climateaudit, but if the oil and coal industry start to offer me money, I'll know I've gone too far :-)
They haven't offered me any money, so obviously I'm doing something wrong.
Actually what I'm doing wrong is not cosying up to the environmental lobby, because that's where the oil companies are at.
We hear lots of times of the few tens of thousands of dollars donated by Exxon to tiny groups (so that anyone remotely connected to them becomes instantly and permanently slimed), but never hear mention of the millions given by Enron to Greenpeace (which they refused to give back).
Funny that.
Dear James
"the realclimate contributors have posted somewhat similar comments ..."
you get no validation from realclimate; it is your views that matter.
"if the oil and coal industry start to offer me money, I'll know I've gone too far :-)"
the unmistakable innuendo here is that industry is corrupt, and pays people money so that they go "too far".
In fact, industry has got a great deal to gain from following good science. After all, bad science can always show industry to be bad; but if industry follows good science, it will at least have a rational basis on which to proceed. There are of course examples of corruption and bribery in industry, but your crude depiction of an evil empire is thoughtless.
yours
per
What's so bad about the oil industry? I think it's pretty cool. Huge capital investments with decadal payoffs. Spudding in in desolate war-torn places. Going to work in blue jeans. Macho almost all male environment. Many advances in engineering and material science driven by the needs of deep drilling.
Frigging kick ass cool. No?
P.s. You said no serious discussion at the header...so I was worried that my earlier comments on philosophy of journal publishing might be crossing the line. I'll troll from now on. Right now, we are at half glass of wine level (after lifting legs...ooh-rah!) level of teasing.
TCO,
Sure there are fun (and even good) things to do in industry. Forecasting oil reserves has a certain amount in common with climate forecasting, for instance. But the only interest that such companies seem to have in long-tem climate change is in paying people to obfuscate mendaciously, which doesn't sit well with my mission to be a beacon of rationality and clarity in the fog of what passes for debate these days :-)
BTW, note that I expect rather higher levels of discourse on my blog, than might be tolerated in Some Other Places :-)
For those of us who have only followed science from the outside and are not involved day-to-day, the rise in blogging by scientists has opened up aspects normally only seen briefly in science histories. While an eye-opener initially, it does help to see the whole process as more human.
It seems some people are still having problems resolving this and they think that what is (seemingly) par for the course, undermines the whole edifice...whereas I would suggest (though perhaps not too strongly) that it reaffirms it.
the only interest that such companies seem to have in long-tem climate change is in paying people to obfuscate mendaciously, which doesn't sit well with my mission to be a beacon of rationality and clarity in the fog of what passes for debate these days :-)
well, you don't get much more clarity and rationality than that !
yours
per
James- You have identified a critical issue with respect to the scientific literature that concerns all of us. The filter of a scientific paper must be expeditious peer reviews, with all papers reviewed that are of an appropriate topic for the journal that they are submitted to. For the Editor to make a unilateral decision to reject a paper compromises the review process.
When I was Chief Editor of the Monthly Weather Review, and co-Chief Editor of the Journal of Atmospheric Science, all topic appropriate papers were reviewed. Journals who do not permit all papers to be reviewed, unforunately, limit the scientific exchange of information.
Whether, James, RP Sr. or I like it or not, Science/Nature/PR Letters are not going to change.
As for the oil business, they may have been mighty investors 40 years ago, but today almost all of the production is controlled by national oil companies, and any investment made by the seven sisters (three have passed on to the great merger in the sky) has a lot of sovereign risk. As a matter of fact, they are pretty risk adverse these days as you can see from their getting out of the refining business. Easier to coin money.
Oh, I don't expect them to change. But I do reserve the right to tell them that they are doing a disservice to their readers and the scientific community in general by putting a near-insurmountable barricade in the way of any possible criticism of the stuff they publish. It's not as if a "comment" is even competing for space, as such articles only appear on the web anyway.
It's worse than that, they are by nature private clubs and the non-U are not allowed. To an extent publishing there is an exercise in Groucho Marxism and except for denial of a boost to your career it would be funny.
Update?
BTW, a long paper sounds great. And I've always pushed Steve to do more peer-reviewed papers.
Update?
Um...the Hegerl et al comment is in the bin, the Frame et al comment is still in play (revised version awaiting reply/review) and I'm trying to do some productive work for a change :-)
There is definitely stuff that needs to be said. I'm not sure I'm the best person to say it (no: I'm sure I'm not the best person to say it), but if no-one else is going to...
Please say it. Even if it's not perfect, it still helps to say it. And the process of saying it will make you more perfect. Do your best to make the saying perfect, but don't let the fear of not being perfect, prevent the saying.
Science needs publications. It is a waste to have a result and not to publish it. If you are government funded, it is a waste of tax dollars. If you can'g be compelled to publish widely for personal reasons, do so for altruistic reasons.
"how does quality science result? Is it like magic?"
"Basically, yes it is :-) After all, your computer works, planes fly, medicines are effective - despite the fallibility of the people who research such things."
I would say that you are talking about engineering here, not science (disclosure: I am an engineer). Put simplistically: A paper can be published if you convince your peers it is right; An airplane flys only if you get it right. Peer review is the least of an engineer's worries.
I'm not suggesting peer review has no place -- I wouldn't want Nature to start demanding working prototypes. Most good ideas start out as speculation, and the peer-reviewed literature is a great engineering resource -- it just needs to be carefully verified.
Post a Comment