No, don't answer that :-)
By now probably many of you will have seen the discussion regarding this manuscript. Roger Pielke Snr covers the story here and Fergus has added some commentary here and here. For those of you who are interested in how such an odd triple came to co-author the paper, I'll go into that at the bottom. But first to flesh in some of the details of the submission process:
We first sent it to EOS for their Forum section, which as Roger says, it seems well suited for. Note that this is not a formally peer-reviewed publication in the way that most academic journals are - it is a newspaper, not a research journal (their own choice of words). The manuscript sat on the editor's desk for an astonishing 4 months, and Fergus' occasional polite enquiries were fobbed off rather abruptly, until eventually we got a brief rejection email from Fred Spillhaus on the grounds that they wanted to focus only on science, not opinions (it took him 4 months to work out that it was an opinion poll?). He has not replied to any further requests for clarification as to how he squares this explanation with the stated policy:
Nature Precedings is basically a non peer-reviewed preprint service (maybe a rival to the Arxiv?) that merely screens for "relevance and quality". So it was rather a surprise to get a one-line rejection that they were "unable to post your document at this time". The email was anonymous and the author(s) did not explain whether it was because they considered the manuscript irrelevant, or rather than it was too poor quality, or both. Only a minuscule proportion of scientific papers get mentioned in the press, and as I've mentioned this has been picked up in a few places despite having no PR, so it is apparently relevant to some. Obviously I'm biased but the quality of the work and presentation seems well up there with the typical middle-of-the-road conference presentation/poster type of thing (remember we aren't talking high-impact peer-reviewed journals here, just a preprint server).
So it seems that no-one wants to publish it, and no-one wants to say why...with Fergus moving on to other things, it seems like we are at a dead end.
As for my participation in this:
The first I heard about it was an email from Fergus (who I know via his blogging, but not otherwise) asking for comments on his proposed poll. I was generally supportive of the idea and offered some suggestions on the questions and format. I also participated in the poll (FWIW I was a 5: although I can see some arguments leaning towards 4 and 6, they are IMO not strong enough to justify actually choosing one of these options, even as a half point). Later on, he sent me the manuscript again asking for comments, and I suggested some edits. It was around this point that the question of co-authorship was mentioned, and although my contribution had been rather minor the other two seemed keen to include me and I was happy to accept. I have certainly known co-authors do less work (though not on papers where I was first author)!
One can always quibble over details of the wording, but IMO the questions are clear enough, the set of scientists polled is very reasonable and carefully controlled (due entirely to Fergus' hard work) and the results are written up fairly and accurately. Indeed I think it stands in striking contrast with the previous survey of Bray and von Storch, where the questions were more ambiguous (how much do you agree with the sentence "Climate change is mostly the result of anthropogenic causes"?) and the survey was open to anyone who found out about it, including the entire readership of the "climate sceptics" mailing list. Of course the main weakness is in the response rate of ~10%: that leaves open the possibility that the 90% non-responders were either all firmly suportive of the IPCC and saw the poll as a bit of irresponsible trouble-making that didn't justify a response, or all so thoroughly alienated and marginalised by the IPCC that they don't have the energy to grumble about it. Personally, I think the first of these is much closer to the truth, but it seems we will never know for sure. Of course, all surveys suffer from this problem to some extent. I bet all the current polls on Clinton vs Obama have enough refusals to completely dominate the result, were they all to end up on one side of the fence. Yet you don't see reports saying "Clinton 22%, Obama 24%, and the other 54% slammed the phone down".
By now probably many of you will have seen the discussion regarding this manuscript. Roger Pielke Snr covers the story here and Fergus has added some commentary here and here. For those of you who are interested in how such an odd triple came to co-author the paper, I'll go into that at the bottom. But first to flesh in some of the details of the submission process:
We first sent it to EOS for their Forum section, which as Roger says, it seems well suited for. Note that this is not a formally peer-reviewed publication in the way that most academic journals are - it is a newspaper, not a research journal (their own choice of words). The manuscript sat on the editor's desk for an astonishing 4 months, and Fergus' occasional polite enquiries were fobbed off rather abruptly, until eventually we got a brief rejection email from Fred Spillhaus on the grounds that they wanted to focus only on science, not opinions (it took him 4 months to work out that it was an opinion poll?). He has not replied to any further requests for clarification as to how he squares this explanation with the stated policy:
Forum contains thought-provoking contributions expected to stimulate further discussion, within the newspaper or as part of Eos Online Discussions. Appropriate Forum topics include current or proposed science policy, discussion related to current research in our fields especially scientific controversies, the relationship of our science to society, or practices that affect our fields, science in general, or AGU as an organization. Commentary solely on the science reported in research journals is not appropriate. [my italics]But with all emails to him simply vanishing into a black hole, it soon became clear there was no point in pursuing that route any further. Anyway, by this time the survey results had been spotted by some sharp-eyed journalists, and it was getting mentioned in various places (such as here and here). So Fergus then approached the Nature Climate Feedback blog, asking if they were interested. Olive Heffernan replied that they weren't open to guest bloggers, but that he should send it to Nature Precedings and after it had appeared there she could write about it herself.
Nature Precedings is basically a non peer-reviewed preprint service (maybe a rival to the Arxiv?) that merely screens for "relevance and quality". So it was rather a surprise to get a one-line rejection that they were "unable to post your document at this time". The email was anonymous and the author(s) did not explain whether it was because they considered the manuscript irrelevant, or rather than it was too poor quality, or both. Only a minuscule proportion of scientific papers get mentioned in the press, and as I've mentioned this has been picked up in a few places despite having no PR, so it is apparently relevant to some. Obviously I'm biased but the quality of the work and presentation seems well up there with the typical middle-of-the-road conference presentation/poster type of thing (remember we aren't talking high-impact peer-reviewed journals here, just a preprint server).
So it seems that no-one wants to publish it, and no-one wants to say why...with Fergus moving on to other things, it seems like we are at a dead end.
As for my participation in this:
The first I heard about it was an email from Fergus (who I know via his blogging, but not otherwise) asking for comments on his proposed poll. I was generally supportive of the idea and offered some suggestions on the questions and format. I also participated in the poll (FWIW I was a 5: although I can see some arguments leaning towards 4 and 6, they are IMO not strong enough to justify actually choosing one of these options, even as a half point). Later on, he sent me the manuscript again asking for comments, and I suggested some edits. It was around this point that the question of co-authorship was mentioned, and although my contribution had been rather minor the other two seemed keen to include me and I was happy to accept. I have certainly known co-authors do less work (though not on papers where I was first author)!
One can always quibble over details of the wording, but IMO the questions are clear enough, the set of scientists polled is very reasonable and carefully controlled (due entirely to Fergus' hard work) and the results are written up fairly and accurately. Indeed I think it stands in striking contrast with the previous survey of Bray and von Storch, where the questions were more ambiguous (how much do you agree with the sentence "Climate change is mostly the result of anthropogenic causes"?) and the survey was open to anyone who found out about it, including the entire readership of the "climate sceptics" mailing list. Of course the main weakness is in the response rate of ~10%: that leaves open the possibility that the 90% non-responders were either all firmly suportive of the IPCC and saw the poll as a bit of irresponsible trouble-making that didn't justify a response, or all so thoroughly alienated and marginalised by the IPCC that they don't have the energy to grumble about it. Personally, I think the first of these is much closer to the truth, but it seems we will never know for sure. Of course, all surveys suffer from this problem to some extent. I bet all the current polls on Clinton vs Obama have enough refusals to completely dominate the result, were they all to end up on one side of the fence. Yet you don't see reports saying "Clinton 22%, Obama 24%, and the other 54% slammed the phone down".
8 comments:
Interesting. Certainly far from crap, in fact quite good, and since an average 5 score seems to concur with the IPCC, I wouldn't think too hot to handle either.
As for the survey itself, I wonder if the close consensus amongst the 140 scientists with the IPCC position represents wide support for current opinion, or if those holding to the current opinion, albeit anonymously, were the only ones willing to put their heads above the parapet.
This raises the question as to the opinions of the other 1667 who didn’t reply. All climate change deniers wishing to hold onto their research grants? Obviously not, but it would be interesting to know.
Well, have you tried Energy & Environment?
Failing that, the last chance would be Tierney's NYT blog.
But seriously --
I'd guess the main problem is the low response rate. I haven't done polling since the late 1960s, but getting enough responses to have a hope of a valid sample was important back then and likely still is.
Is there any way to get the organizations involved to formally poll their members?
I realize the lack of responses to the NYT dot.earth policy statement thread is discouraging in that regard, but the organizations weren't asked to invite their members to that, it was just word of mouth and likely that's why they got so few actual members replying.
I mentioned it to Fergus a long time ago, but let me repeat it here.
Polling is a field like climate science. Everyone thinks they can do it, and mostly they do. . . badly. You really needed help in designing the sampling and response parts.
Other than that, I was not too happy with the questions. I thought a number were subject to interpretation, but what do I know about polling
I think it's a fairly decent poll, at least compared to previous attempts as you mention. I do agree with Eli though that some questions are open to interpretation: Eg what is the difference between "natural" (option 2) and "within natural limits" (option 3)? And option 6 and 7 say mostly something about the need for action, but both statements can easily be reconciled with option 5 at the same time. Option 6 is perhaps even stronger worded than option 7 ("compromised by political intervention"). I would have a hard time chosing between options 5, 6 and 7.
I partly disagree with your interpretation of the results. In framing the replies as 15-20% finding the IPCC too cautious or understating the problem, and 15-20% finding the IPCC too certain or overstating the problem, you suggest that 15-20% of the scientists would be in the “skeptics” camp. While option 4 is mildly critical of the IPCC, but doesn’t necessarily imply that warming is either predominantly natural or not problematic. I find a major conclusion from this poll that only a very small fraction (of 4 or perhaps 7% (if option 3.5 is included) attribute a substantially larger proportion of the warming to nature, and can as such be considered “skeptics”. Option 4 includes the criticism that the non-CO2 greenhouse gases and changes in landuse (Pielke’s favorite) are more important than IPCC implies.
Dr Annan, you note the low response rate (<10%). Perhaps, one way to judge the response bias is to see whether there is any difference between those respondents who quickly replied to the email and late replies. It may give some answer to whether "non-responders were either all firmly supportive of the IPCC and saw the poll as a bit of irresponsible trouble-making that didn't justify a response, or all so thoroughly alienated and marginalized by the IPCC that they don't have the energy to grumble about it."
Interesting idea, but I don't actually have any of the data, it is all in Fergus' hands. I'm not sure that there is any reason to think that either set would reply faster - I'd expect that people either replied pretty quickly or just forgot/ignored it.
The point is that later respondents are more likely to the representative of non-respondents. Ergo, if later respondents tend to be more critical of the IPCC than early respondents, then it follows that non-respondents are more likely to be critical as well, and so on.
The survey seemed well done (from my humble perspective), and confirmed what I already thought - that the IPCC was the "mean response", and that more extreme views from both sides were left out somewhat equally.
However, certain parts of the article - especially the abstract - could be easily taken out of context. "There is not a universal agreement among climate
scientists about climate science as represented in the IPCC's WG1" and "there
remains substantial disagreement about the magnitude of [climate change] impacts" would immediately be interpreted by the public as "the IPCC is overstating climate change".
In newspapers, you hear a lot from people who think the IPCC overstates, but almost nothing from those who think it understates. Therefore, "substantial disagreement", without an explicit note that the disagreement was both for under- and over-stating in the IPCC, would seem to the public to be exclusively the latter.
And of course those climate denial blogs would lap it right up. The Heartland Institute would be citing you for years.
Add one or two sentences to the abstract, and you'd avoid that angry blogging storm and a whole host of public misconceptions.
Post a Comment