Credit where it's due

The GMD t-shirts continue to be good value. Jules and I usually try to wear ours at any scientific meetings to spread the word and it often sparks off some interaction. Recently someone prominent came up to me at the workshop dinner and congratulated me on my achievements in respect of GMD. He was particularly impressed with the CMIP6 special issue which is turning out to be very useful. I pointed out that jules (who was sitting beside me but in civvies) actually bore far more responsibility for this, not only in being Chief Exec Editor of the journal but also more specifically both in developing the whole concept of MIP papers within GMD and also in negotiating the details of how the CMIP6 special issue would work - which involved some lengthy negotiations with CMIP peoples.

At the end of the dinner as he was leaving, he thanked me again for all that I'd done. It's a tough job taking credit for other peoples' work but someone's got to do it!

Apocalypse now?

Lots of people have asked about this paper (which I think is open access).

To cut a long story short, it's not silly - the authors are entirely respectable and the work is interesting - but I don't think it is really that credible in terms of overturning established consensus. In fact it looks to me like they've gone astray in a few ways which add up to provide plenty of reasons for doubting the result.

The underlying idea is interesting enough and I have no problem with it in principle. They looked at climate change over multiple glacial cycles, to estimate not only the climate sensitivity, but also tease out how much this varies with temperature. Their observed temperature record comes from a handful of long-term proxy records of sea surface temperature, just 14 in total, which do not give very good global coverage. So they start by calibrating these records to a global mean temperature by comparing the local (proxy location) to global temperature at the last glacial maximum as simulated by models. The LGM temperature change arising from their 14 proxy records scaled to global temperature is about 5C colder than pre-industrial. This is a fair bit colder than the 4C we got with 400 data points over both ocean and land. But not content with this, they then average it with the mean of the PMIP model simulations, which is 6.5C colder than PI, thus getting a cooling of almost 6C.

Edit: Thanks to an email from Axel, I've had a more careful read and the above is wrong. One estimate is the PMIP models scaled to match data ("proxy-based"), another is their LOVECLIM simulation scaled to match a different data set ("model-based").
 
It is probably defensible to use the PMIP models in this way as some sort of independent estimate of the LGM state, but surely it is inconsistent to not then also use the PMIP models to estimate the cimate sensitivity and/or its nonlinearity. Anyway, this cold LGM state feeds through into a high sensitivity. An important additional factor here is the nonlinearity which they diagnose by comparing temperature to net forcing throughout the time series. I think a fair bit of this nonlinearity relates to the very high interglacials which are at best poorly calibrated since they only calibrated the proxy records to a fully glacial state. Interglacials have much smaller global temperature signal compared to the present, with the regional differences being much more important, and it seems doubtful whether a single scaling applied to these 14 proxy records could represent the true relationship with adequate precision for their purposes.  In support of this, the last interglacial appears to have extremely high warmth in their calibrated proxy record of some 3C above pre-industrial, which I don't think is widely accepted. On the other hand, some nonlinearity is probably quite plausible, so let's press on. Using the "warm" sensitivity of 4.9C/doubling, they then generate a transient prediction, using a simple energy balance with the ocean heat uptake factor again taken directly from the CMIP models.

Disappointingly, their plot of the transient warming from 1880-2100 doesn't show the actual observations up to the present. It is hard to be precise from eyeballing a computer screen, but it looks to me that their new improved prediction is already way ahead of observations. It suggests a warming that first reaches 1C (relative to 1880) back in the early 1990s before Pinatubo, rebounding from that brief dip to reach about 1.5C by the present. HadCRUT4 shows rather less warming that this, with even the current extraordinary hot year (boosted by a strong El Nino) not reaching 1.2C on that anomaly scale. In my view failing to show, or discuss, this discrepancy is a major failing of the paper. If they think it can be explained by internal variability then they should have presented that discussion and I'm surprised the reviewers let them get away without it.

Edit: ok, here is a very quick and dirty attempt to show what their pic would have looked like with real temperatures on it:

Not a great graphic, I just scaled the hadcrut pic off here and tried to line it up with the authors' own axes, matching the baseline temps around the end of the 19th century.  As anticipated, recent years are well below their prediction, with 2016 just about reaching the CMIP mean.

Edit: Axel claims that internal variability can explain this discrepancy, but I don't believe it. The magnitude of decadal-scale internal variability is about 0.1C (Watanabe et al 2014 and Dai et al 2015) and this new forecast would be even hotter if it wasn't also hugely overestimating the response to volcanoes.

So, in summary, nice try, but I don't believe it, and I don't really think the authors do either.

[Blog post title inspired by the Mark Lynas quote which is not the authors' fault. Incidentally, it is disappointing to see journalists falling for the parasitic publishing scam in which "one of the most respected academic journals" cashes in on its name by setting up numerous sister journals which share some elements of the name but neither the editorial policy nor barriers to entry. "Science advances" is not Science and it's only been around for a year or so, nowhere near long enough to have any sort of reputation. But if journalists don't know the difference, scientists will happily pay the steep publication charge and reap the publicity benefits. Nature have been doing this for a few years now (eg Nature Communications) so it's hardly surprising Science have followed suit.]

Open review vs open access.

Now that Boris Johnson has finally Godwinated the EU referendum debate I don't need to discuss that, and can talk about this instead. Though actually, it's about at the same intellectual level as Johnson's comments. Everyone agrees it's junk, the questions are why this stuff gets written, and how it gets published. I won't speculate here on the why (I already did here), just the how.

It has appeared in a new journal of unknown quality, the AGU (partnering with Wiley) seemingly jumping on the open access pay-to-publish bandwagon. It is worth noting, Bates has not previously published there, he didn't choose it out of familiarity or convenience, but had (apparently) been shopping around since at least 2014 with this idea trying to find a home. Pretty much everything gets published eventually, by the way. It just takes longer if it's either rubbish, or a revolutionary idea that is well ahead of its time.

It's a shame Bates didn't have the guts to try ACP or ESD, where his manuscript would have been shredded by reviewers in short order. It is all too easy for a lazy or overstretched editor at any journal to simply use the author's “suggested reviewers” without also looking for an independent view. At least in the case of the EGU open review process, an incompetent reviewing process is in principle discouraged by the fact that it's out in the open, and there is of course the opportunity for other interested parties to add their views. Score +1 for open review.

New publication model for EGU journals

There's been longstanding debate over the way in which manuscripts are handled by the EGU/Copernicus journals. The current system is that publication fees are charged for submitted manuscripts, which are professionally typeset (hence incurring significant costs) and appear in an on-line “Discussion Journal” like CPD, GMDD etc where they remain as part of the public record (with DOI etc) irrespective of whether a final peer-reviewed paper appears or not in the non-D journal. There are several drawbacks with this approach, though its originators and advocates have always argued that the benefits outweigh these.

Well, it's all changing now anyway. From January, submitted manuscripts will not be professionally typeset, the Discussion journal will cease to exist (though the submitted manuscripts will still be public and citeable), and paper charges will only be applied after the paper has been through peer-review. It will be interesting to see how this affects the journals, and whether for better or worse. As for me, I'm pretty ambivalent about it. Paying charges in advance was always a bit awkward and puts editors in a difficult position when rejecting papers (at GMD, the unofficial policy was to offer a hefty discount on fees for their next submission). On the other hand, perhaps it does dissuade a certain amount of dross - though it apparently didn't help with this nonsense. Anyway, the new system will be much closer to the standard model for journal publication, while retaining the special feature that not only the final manuscript, but also the review process, is on-line and open to all.

GMD raises the bar again

Not content to rest on their laurels, the executive editors at GMD have revised and updated their policies, the new version of which can be found here. Of particular note, it is now a requirement that code must always be made available to the editor at minimum, and each manuscript must include a paragraph outlining code availability. (In practice we expect that most model code will be publicly available.) This represents a logical and incremental change to the previous policy, and once again sets a new standard for open and accessible science.

It is increasingly common for journals to have code and/or data availability policies, whether or not they are enforced. This BAMS paper is also relevant, focussing on the analysis code rather than numerical model itself. 

(Disclaimer: I helped to found GMD and was an executive editor for several years. However I no longer hold this position though I continue to act as a topical editor [which basically means I handle manuscripts but don't set the editorial policy].)

Peer review as a business transaction

Amusingly hidden behind a paywall as usual (ironic considering that Nature bases its content and profit on the free labour of others, including the reviewers), there's a letter suggesting peer reviewers should be paid. I've found this idea a handy way of turning down reviewing invitations that I'm not really that desperate to do, without appearing too rude or selfish. "Please Dr Esteemed Scientist, we would greatly value your expertise in reviewing this manuscript." "Yes, of course I'd be happy to review it, and my fee will be £X." "Oh, we don't really think your input is valuable after all. " Kthxbye.

Spam spam lovely spam

I'm sure most people get enough unwanted email that they don't really want to share mine, but this solicitation for a junk journal was more amusing than most. 

As well as the imaginative and tasteful use of colour in the email, I was particularly impressed by the number of digits they quote for the citation statistics. Wouldn't do to confuse this journal for one with a crappy impact factor of only 0.100!

------------------------------------------------------------------------------------------------------------

Call for Papers Submission

******************

Mitteilungen Klosterneuburg

http://mitt-klosterneuburg.com/

ISSN: 0007-5922

Peer reviewed highly cited

ISI Indexed Journal

As a bimonthly interdisciplinary life science journal, we aim to publish scientific papers in all fields of the human and natural sciences.

The mission of the Journal is to promote excellence in leadership practice by providing a venue for academics, students, and practitioners to publish current and significant empirical and conceptual research in all fields of sciences. The journal covers a very wide range of areas and we welcome submission from researchers at all levels and from all over the world.

Publisher:

HOEHERE BUNDESLEHRANSTALT UND BUNDESAMT FUER WEIN- UND OBSTBAU

Impact Factor: 0.106
Immediacy Index: 0.019
Eigen factor TM Score: 0.00007

Another journal editor resigns!

Regular readers will have noticed that I follow the goings-on at EGU journals with some interest. So in that vein I'd like to point out there have been some recent changes at GMD. Perhaps most notably, our Dear Leader Dan Lunt has stepped down from the position of Chief Executive Editor, which he has held since the journal's inception about 6 years ago. Jules is the incoming chief. (Chief doesn't actually have any extra powers that I'm aware of, but is expected and trusted to take the lead on many decisions with or without discussion.) Bob Marsh has been added to the list of execs - this happened last year actually - having been a topical editor for some time. And...drum roll...I am no longer on the list of execs, though I'll remain a topical editor. All the execs feel that the journal (indeed all EGU journals) should be regarded as community assets rather than personal fiefdoms. So although it made sense to stick with a core team who shared a clear vision though the early years, we realised some time ago that it was time to bring in new ideas and let things evolve a bit. This feeling has been informally formalised though a rough plan to swap execs off the board on a biannual biennial every two year basis - Bob's induction was the start of this, staggered with my resignation to allow a bit of settling in time - and also rotate the chief exec position among the board members. I'm happy to leave the journal in the capable hands of the new board.

Incidentally, it is rumoured that the new Impact Factor for the journal will be approaching 6, up from 5 last year. That should put us even closer to the top of the list for journals in the geosciences! I'm sure that GMD, and all the other EGU journals, will continue to go from strength to strength as the open access movement continues to gain momentum.

Is for-profit publishing dead?

I'm talking about academic journals, of course. People have been grumbling for ages about what a scam it is: academics not only provide the material for free but even pay the journal handsomely for the privilege of appearing in their hallowed pages, then other academics review and edit the material for free, and finally yet more academics pay to be allowed to read it. In these days of electronic transmission and powerful searching and indexing facilities, the added value provided by the publisher appears limited at best, and indeed strongly negative when you consider the sums of money involved. So I am following this initiative with some interest. I haven't signed up yet because I haven't checked how many relevant journals are run by Elsevier, nor whether they are a particularly egregious offender.

As an example of what it actually costs to run a professional outfit, the EGU journals generally charge a publication fee of €24 per "page". The "page" size for this calculation is unusually small (1900 characters), but a typical paper is still only around €600 in my experience, and a previous short comment was the princely sum of €170. This is the total price, after which the resulting paper is open access. Compare to AGU journals, where the basic fee is already often larger (GRL is $500 for 4 pages, and longer JGR papers $1000) even when the final article is paywalled, and the open access option is another $2500-$3500 on top - an obvious indication of the value of our work that we have simply been giving away. I suppose at least with the academic societies the profit might be going to reasonably good causes (such as funding the sterling work of their Ethics Task Force, I presume), but even with their low fees the EGU publication empire runs at a modest profit level, sufficient to fund gradual expansion.

Previously I have grumbled about the EGU stable of journals having a gaping hole in that papers on climate change and prediction - surely one of the largest constituencies of their community - do not have a suitable home (CP and ACP have taken a few climate change papers, but the fit seems tenuous). However, a couple of years ago, they started up "Earth System Dynamics". This looked initially according to its blurb like it might be a bit too Earth-systemy for my tastes, but it's also had a steady stream of more conventional (primarily physical) climate change research. One paper that looks like it could kick off a bit of a bunfight is this one here about diagnosing climate feedbacks from satellite measurements, (which refs previous Spencer+Braswell vs Dessler spats) for which Andrew Dessler was quick to get in a somewhat critical review. I haven't looked at the manuscript carefully enough to judge for myself. Anyway, I'll certainly bear the journal in mind for future manuscripts, and hopefully it will take some business away from the leeches.

Incidentally I think a commenter previously pointed to this paper some time ago, but I can't find where. It seems that enough reviewers noted the somewhat woolly logic behind the climate sensitivity calculations that it did not progress to the final peer-reviewed publication stage. There is also this manuscript which got somewhat similar comments.

Wiley Interdisciplinary Reviews: Climate Change

A new journal has sprung up recently, I'm not entirely sure why or how, but it seems to be open access for now (not indefinitely) and has some interesting papers so maybe some of you would like to take a look. Called "Wiley Interdisciplinary Reviews: Climate Change" it seems to be a cross between an interdisciplinary journal and collection of encyclopaedic articles on climate change. There are a number of other WIREs journals on unrelated topics such as computational statistics, and nanomedicine and nanobiotechnology.

The Editors seem a bunch of slightly unconventional people, a little removed from the mainstream IPCC stalwarts though eminent enough and with some IPCC links: Hulme, Pielke, von Storch, Nicholls, Yohe are names that many will be familiar with. The others are probably all famous too, but I'm too ignorant to recognise them. I'm sure the journal is not intended as a direct rival to the IPCC, but it may turn out to provide an interesting and slightly alternative perspective.

The articles to date include a mix of authoritative reviews from leading experts - such as Parker on the urban heat island, Stott on detection and attribution, interspersed with perhaps more personal and less authoritative articles. I can safely say that without risk of criticism because one of them is mine - a review on Bayesian approaches to detection and attribution. This article had a rather difficult genesis. I was initially dubious about my suitability for the task and indeed the value of the article, but after declining once (and proposing another author, who also declined) I changed my mind and had a go. My basic difficulty with addressing the concept is that D&A has always seemed to me to be a rather limited and awkward approach to the question of estimating the effects of anthropogenic and natural forcing, which is tortured into a frequentist framework where it doesn't really fit. Eg, no sane person believes these forcings have zero effect, so what exactly is the purpose of a null hypothesis significance test in the first place? However, conventional D&A has such a stranglehold on the scientific conscious that most Bayesian approaches have actually mimicked this frequentist alternative of the Bayesian estimate that you really wanted in the first place. It all seems a bit tortured and long-winded to me.

Anyway, I eventually found some things to say, which hopefully aren't entirely stupid and help to show how a Bayesian approach might actually be useful in answering the questions that (sensible) people might want to know the answer to, rather than the relatively useless questions that frequentist methods can answer, which are then inevitably misinterpreted as answers to the questions that people wanted to answer in the first place (as I argue and document in the article).

Another of the personal and argumentative articles was contributed by Jules, who was invited to say something about skill and uncertainty in climate models. This was actually the article that sparked off our "Reliability" paper, as our discussions kept coming back to the odd inconsistency between the flat rank histogram evaluation that I know is standard in most ensemble prediction, versus the Pascal's triangle distribution that a truth-centred ensemble would generate (ie, if each model is independently and equiprobably greater than or less than the observations, then the obs should generally be very close to the ensemble median). Of course this problem didn't take long to solve once we had set out the issue clearly enough to recognise that there really were two incompatible paradigms in play, and Jules even ended up citing the GRL paper which overtook her WIREs one in the review process.

Perhaps of more widespread interest to other readers, is a simple analysis of the skill of Hansen's forecast which he made back in 1988 to the US Congress. We'd actually had lengthy discussions with several people (listed in the acknowledgements) a year or two ago, trying to resurrect the old model code that was used for this prediction in order to re-run it and analyse its outputs in more detail. But this proved to be impossible. (The code exists but has been updated and gives substantially different results. If only the code had been published in GMD!) Therefore we were left with nothing more than the single printed plot of global mean temperature to look at. This didn't seem much to base a proper peer-reviewed paper on, so the idea died a death. When this WIREs invitation came long it seemed like a good opportunity to publish the one usable result we had obtained, as an example of what skill means. The headline result is that under any reasonable definition of skill, the Hansen prediction was skillful. While no great surprise, I don't think it has been presented in quite those terms before. It's a shame that we weren't able to generate a more comprehensive set of outputs though which might have given a more robust result than this single statistic.


The null hypothesis of persistence (no change in temperature) was found to give best performance over the historical interval, compared to extrapolating a trend. So this is the appropriate zero-skill baseline for evaluating the forecast. Nowadays with the AGW trend well established, probably most would argue that a continuation of that trend is a good bet, though that still leaves open the question of how long a historical interval to fit the trend over. Anyway, the model forecast is clearly drifting on the high side by now - most likely due to some combination of high sensitivity, low thermal inertia and lack of tropospheric aerosols - but is still far closer to the observations than we would have achieved by assuming no change. Furthermore, the observed warming is also very close to the top end of the distribution of historical 20 year trends, meaning that the observed outcome would be very unlikely if the the climate was merely following some sort of random walk. This evidence for the power of climate models is obviously limited by lack of detailed outputs for validation, but what there is is clearly very strongly supportive.

Govt policy "reprehensible" says Science and Technology committee

Oh, but they didn't say it in quite that way, instead preferring to portray it as a fault of the "climate science community":

"The parliamentary science and technology select committee was scathing about the 'standard practice' among the climate science community of not routinely releasing all its raw data and computer codes – something the committee's chair, Phil Willis MP, described as 'reprehensible'. He added: 'That practice needs to change and it needs to change quickly.'"

Let me introduce you to the NERC policy on Intellectual Property. Short version: "Who owns the intellectual property? We do." The UK Ministry of Defence (who run UK Met Office and therefore the Hadley Centre) is orders of magnitude worse in its defensive and bean-counting approach to the supply of, well, just about anything that they have and anyone else wants. The bottom line is (or certainly was, when I worked there) that NERC employees are under pressure to sell anything that can be sold. And if someone asks for something, that means it must surely be worth something, right? Of course this is an attitude that the scientists - who know that they can't really get any significant price for their work - have always implacably opposed, but we don't really count for much when the politicians are demanding budget cuts and percentage returns on investment.

[In fact when I was a NERC employee, they once tried to modify our contracts in order to claim IP rights over everything we ever did, even if entirely unrelated to our jobs, such as taking beautiful photographs as a hobby or composing pop songs in the bath. Many of us refused point-blank to sign up to such absurd terms, there was enough opposition that even the union was persuaded to oppose it, and the idea was eventually dropped.]

Occasionally, it works ok: a previous employer sells this software to mariners all over the place, and I believe that the commercial department actually brings in more money than it costs to employ them (at least that was the situation back then). The vast majority of our work has no meaningful commercial value, however, but the effort of demonstrating this may not always be trivial.

I don't defend any unnecessary secrecy: I was disappointed when a prominent climate scientist refused to allow me to have his widely-used code (a Windows executable was available, but that was no use to me) so I used another model and the developers have gained a handful of highly cited publications as a result. I am also one of the founding executive editors of Geoscientific Model Development, "an international scientific journal dedicated to the publication and public discussion of the description, development and evaluation of numerical models of the Earth System and its components. " We explicitly encourage and support the archiving of code to support the paper.

Returning to the politicians:

""It is important in terms of scientific endeavour that that material is made available," said Willis. He added that the committee accepted that Jones had released all the data that he was able to."

Well, quite. There's no point in MPs pointing fingers when they are the ones setting the policies that make it impossible (or at least difficult and deprecated as a general principle). Is it possible they don't even realise this?

Frogs and blogs

I was going to write about the frogs, but the longer I delay the more stuff comes up and now this is going to be a rather lengthy post...I'll start at the beginning and see where I end up.

Via John Fleck, I see that the wheels of science have finally turned, albeit exceeding slow...

You may recall this story from 2 years ago - someone claimed that scary global warming was killing all the poor frogs. Well, now a mere two years later, other researchers have published a paper which argues strongly that climate change is at most a minor player in this process. That seemed pretty clear right at the start, to some people at least. The lesson here is that yes, science does tend to be self-correcting, but it can be a painfully slow process, and an over-hyped story will be round the world years before the more nuanced version has got its boots on. The Pounds paper has been cited by 155, including the IPCC (uncritically). This is hardly an isolated example, either (eg see also Bryden) - John Fleck has coined the phrase "the Nature effect" for cases such as this.

It is IMO notable (and this is my second point) that the paper attracted intelligent and pertinent criticism in the blogosphere immediately after publication, well before the wheels of the peer-review process started to turn. Those two blogs I linked to are both people I've had plenty of disagreement with in the past (and probably will do in the future), but as I said at the time, that doesn't mean I will jerk my knee and reflexively dismiss anything they say, just because of who said it. Unfortunately WCR rather spoil things with their unthinking endorsement of Chylek's silly paper on paleoclimate and sensitivity - it seems their critical faculties only apply in one direction. So it's definitely a case of reader beware, but the fact remains that the truth certainly can get out there a lot faster in electronic media than through peer-reviewed publication. Of course this relies on some of the bloggers having a clue about what they are writing, which pretty much requires at least some scientists to participate.

So from the POV of my post, it is timely that Nature Geoscience has published a couple of commentaries on the value (or otherwise) for blogging, with Gavin Schmidt on one side, and Myles Allen on the other. There's discussion on RC of course. Gavin puts the case well enough - I can't imagine he found it a taxing task, as the case pretty well makes itself. Given that the internet (and blogging) exists, it is hardly credible for science as a whole to turn its back on this avenue for communication. Myles struggles to pitch blogging against the peer review system, as if they are somehow in competition. Perhaps it's too easy to just say I endorse John Fleck again. So I'll flesh out the point a little with some examples. You may recall the Schwartz paper which claimed to prove that climate sensitivity was very low. Of course it was easy to see substantial problems with the paper. Months later, our Comment is still mired in the review process (it does at least seem that it will be published eventually) but my blog posts have already been repeatedly cited in discussion about the paper, usually to rebut sceptics who are promoting it as the latest proof that global warming is a myth. (Perhaps this can be dismissed as "just the web" but one can't simultaneously argue both that no-one takes the web seriously and also that it actually undermines more traditional media.) So I see no reason to regret, or apologise for, putting the evidence out in the open before it had been peer-reviewed. I also don't feel under any obligation to follow up the rather straightforward and elementary criticism of Chylek (also here) with submitted comments, although I haven't entirely ruled out the idea. Let's not forget that peer review is hardly a faultless process - I don't necessarily disagree with Myles' assessment that it's the best system that we have, but it is well known that it is strongly biased to maintain the status quo, and routinely misses substantive errors. At its best, the careful opinions of unbiased colleagues can greatly improve the clarity and content of the paper, but the minimum threshold may simply be that two of the author's pals (yes, authors generally get to suggest some reviewers, and editors may not have the energy or expert knowledge to go outside that list) glanced over the article and didn't think that it was so bad that it was unpublishable. There is not even necessarily the implication that they think it is right, merely that they think the case is arguable. So let's not get too excited over peer review as some stamp of approval or validity (or conversely, as rejection meaning a paper is necessarily wrong). It's one piece of evidence, of uncertain strength.

As well as dredging up the old "Overselling Climate Change" thing, Myles also gets some digs in at our exchanges. I'm sort of surprised he wants to bring it up again, but I guess he couldn't resist the chance for a few snide comments. Probably I would have done the same had the boot been on the other foot :-)

Unfortunately one commentor on RC has already misinterpreted what he wrote. Where he says "needless to say, our response is not" [available to read as a rebuttal of the criticism], that should not be interpreted, as that reader did, to mean "our adversary is censoring our comments" but rather "needless to say I'd rather my responses are not made public". Of course I'd be very happy to publish the full set of exchanges here, and Dave Frame has already commented several times on my blog with no censorship - I think I even left up all of the hate mail from their colleague Carl Christensen, although I may have deleted some of his more abusive "anonymous" comments. I agree that blogs are a rather one-sided forum (usenet was better, but has been effectively destroyed by trolls) but there's nothing stopping him putting his stuff up on his own web site or the Arxiv or anywhere else he wants to. Alternatively, if he truly thinks that no non-peer-reviewed material should ever be communicated to the public, he could start by taking down the massive amount of non-peer-reviewed stuff up on the CPDN website, and their bulletin board. No, I'm not suggesting that would be a sensible step forward. but it's hard to see how else to interpret

"If, as a scientist, you feel you have to communicate non-peer-reviewed opinions to a journalist or member of the public, then stick to communicating one-to-one and make it clear you are speaking off the scientific record. Better still, don’t"

Hopefully Chris Randles will be along in a minute to explain how I'm misreading that :-)

It's pretty clear that for one reason or another journals don't like to publish comments (eg read Doswell and Errico on peer-review in general and comments in particular). Here's another anecdote I've never blogged about. It is an arithmetic error I spotted in Levitus' seminal ocean heat content paper way back in 2002. It's a just a simple error in a linear regression, so the facts are not in doubt - I was persuaded in the end (see the linked thread) to write a couple of sentences to Science pointing it out but the editor decided, aided by an extraordinary volume of bluster from the original authors, that it was not worth publishing. The data have long since been superceded but it annoyed me at the time that both Science and the authors themselves were quite happy to see a simple arithmetic error (which just coincidentally happened to exaggerate the ocean warming, and showed a rather better agreement with models than the correct analysis would have done) remain uncorrected in the literature and be used in further analyses. (Yes, looking back at that episode I now see I was rather naive at the time, having only recently shifted into climate science. I am sad to say that such behaviour would no longer strike me as unexpected.) But the idea that I should somehow be morally obliged to censor myself because an editor didn't want to devote any space to publishing that small correction is patently absurd. Equally silly is the suggestion that I should not point out the glaring errors in Chylek without feeling obliged to write a short paper. It's a shame that comments seem to be taken as such a personal insult - as I've said before, the only scientist who's never made a mistake is one who's not done any science, although I'm not suggesting that all mistakes merit a published correction (journals would be overwhelmed by trivia). But I know of two recent cases where people declined to co-author a comment due to concerns about offending the (more powerful) recipient. In both cases the potential author was untenured, and one of them explicitly cited that as the major factor in their decision. It's hardly a sign of healthy science when people are too scared to say what they think, on blogs or elsewhere.

Too crap to publish or too hot to handle?

No, don't answer that :-)

By now probably many of you will have seen the discussion regarding this manuscript. Roger Pielke Snr covers the story here and Fergus has added some commentary here and here. For those of you who are interested in how such an odd triple came to co-author the paper, I'll go into that at the bottom. But first to flesh in some of the details of the submission process:

We first sent it to EOS for their Forum section, which as Roger says, it seems well suited for. Note that this is not a formally peer-reviewed publication in the way that most academic journals are - it is a newspaper, not a research journal (their own choice of words). The manuscript sat on the editor's desk for an astonishing 4 months, and Fergus' occasional polite enquiries were fobbed off rather abruptly, until eventually we got a brief rejection email from Fred Spillhaus on the grounds that they wanted to focus only on science, not opinions (it took him 4 months to work out that it was an opinion poll?). He has not replied to any further requests for clarification as to how he squares this explanation with the stated policy:

Forum contains thought-provoking contributions expected to stimulate further discussion, within the newspaper or as part of Eos Online Discussions. Appropriate Forum topics include current or proposed science policy, discussion related to current research in our fields especially scientific controversies, the relationship of our science to society, or practices that affect our fields, science in general, or AGU as an organization. Commentary solely on the science reported in research journals is not appropriate. [my italics]

But with all emails to him simply vanishing into a black hole, it soon became clear there was no point in pursuing that route any further. Anyway, by this time the survey results had been spotted by some sharp-eyed journalists, and it was getting mentioned in various places (such as here and here). So Fergus then approached the Nature Climate Feedback blog, asking if they were interested. Olive Heffernan replied that they weren't open to guest bloggers, but that he should send it to Nature Precedings and after it had appeared there she could write about it herself.

Nature Precedings is basically a non peer-reviewed preprint service (maybe a rival to the Arxiv?) that merely screens for "relevance and quality". So it was rather a surprise to get a one-line rejection that they were "unable to post your document at this time". The email was anonymous and the author(s) did not explain whether it was because they considered the manuscript irrelevant, or rather than it was too poor quality, or both. Only a minuscule proportion of scientific papers get mentioned in the press, and as I've mentioned this has been picked up in a few places despite having no PR, so it is apparently relevant to some. Obviously I'm biased but the quality of the work and presentation seems well up there with the typical middle-of-the-road conference presentation/poster type of thing (remember we aren't talking high-impact peer-reviewed journals here, just a preprint server).

So it seems that no-one wants to publish it, and no-one wants to say why...with Fergus moving on to other things, it seems like we are at a dead end.

As for my participation in this:

The first I heard about it was an email from Fergus (who I know via his blogging, but not otherwise) asking for comments on his proposed poll. I was generally supportive of the idea and offered some suggestions on the questions and format. I also participated in the poll (FWIW I was a 5: although I can see some arguments leaning towards 4 and 6, they are IMO not strong enough to justify actually choosing one of these options, even as a half point). Later on, he sent me the manuscript again asking for comments, and I suggested some edits. It was around this point that the question of co-authorship was mentioned, and although my contribution had been rather minor the other two seemed keen to include me and I was happy to accept. I have certainly known co-authors do less work (though not on papers where I was first author)!

One can always quibble over details of the wording, but IMO the questions are clear enough, the set of scientists polled is very reasonable and carefully controlled (due entirely to Fergus' hard work) and the results are written up fairly and accurately. Indeed I think it stands in striking contrast with the previous survey of Bray and von Storch, where the questions were more ambiguous (how much do you agree with the sentence "Climate change is mostly the result of anthropogenic causes"?) and the survey was open to anyone who found out about it, including the entire readership of the "climate sceptics" mailing list. Of course the main weakness is in the response rate of ~10%: that leaves open the possibility that the 90% non-responders were either all firmly suportive of the IPCC and saw the poll as a bit of irresponsible trouble-making that didn't justify a response, or all so thoroughly alienated and marginalised by the IPCC that they don't have the energy to grumble about it. Personally, I think the first of these is much closer to the truth, but it seems we will never know for sure. Of course, all surveys suffer from this problem to some extent. I bet all the current polls on Clinton vs Obama have enough refusals to completely dominate the result, were they all to end up on one side of the fence. Yet you don't see reports saying "Clinton 22%, Obama 24%, and the other 54% slammed the phone down".

Can you say "survivor bias"?

There's a (presumably unintentionally) funny report on Nature's "Peer-to-Peer" blog today. Entitled "Researchers like the peer-review system", it discusses a study in which "more than 3,000 senior authors, reviewers and editors were asked about the peer-review system". Surprise surprise, they basically liked it (although with some reservations).

Now think about that for a second.

For an example of what I mean by "survivor bias", drawn directly from Peer-to-Peer's previous posting(!), I'd bet dollars to doughnuts that the "more than 3,000 senior authors, reviewers and editors" did not include many of the women who find that they have a much tougher time publishing as first author. Because of course as a result of this gender discrimination, they are less likely to be "senior". One could hardly expect those who have benefited most through the system, to admit that their success is substantially a matter of luck and nepotism, whether or not this is the case. (By "nepotism", I mean that if you want "entry" to the top journals, having a famous supervisor to co-author your papers is very helpful.) One might as well ask sitting British MPs if they support the constituency-based simple majority electoral system that effectively guarantees many of them a job for life (actually, they do).

My ears pricked up when I noticed that the respondents were reported as being hostile to "open peer review". However, on checking the summary, they use this term to mean signed reviews, and it is unrelated to the open system of many EGU journals that I have mentioned before.

[Note that I'm intentionally not saying what I think about the peer review system here. That's not my point. In fact I've had no real problems with peer-review myself, until I started to tread on the toes of some "probabilistic climate prediction experts" at which point my work became simultaneously (1) well-known, (2) interesting and important, and (3) wrong-headed - and most importantly, (4) not appropriate for this journal. But that is a separate rant you've all heard many times. Indeed overall I probably just about qualify as one of those "senior authors" who supports the system pretty much as it is].

Open access publishing

Jim Giles had an article in NewScientist on open access publishing recently, along somewhat similar lines to his article in Nature earlier this year. However, this time it's a comment article rather than mere news, so he's prepared (allowed? Nature's hostility to open-access is hardly a secret) to present his point of view, which is that open access is the way forward. I agree with what he says, although think he misses a detail in his presentation of the matter as one of paper charges versus subscription fees. Many journals charge both! So in fact we are already paying the journals $1000-$2000 to take our work, hide it behind a paywall, and sell it for their own profit.

Since I had it handy, I just checked that the 4-page Comment on Schwartz will cost about $2000 for standard publication (in JGR) assuming some use of colour. In fact I see the AGU has just instigated a new experimental system whereby we can pay the same again (roughly) as an additional charge to have the article made freely available to all readers. So that would make it $4000, just for a short comment. Think I'll pass on that second option, as the AGU (in common with essentially all publishers) do not prevent authors putting papers on their own web sites anyway. Google will usually find the full text for recent and even forthcoming papers these days (old pre-web ones are harder to track down).

The EGU open access journals somehow manage the whole process far cheaper - and I don't think they are heavily subsidised by the EGU itself, at least not in the long term. Their page charges are about €20 per page. Even with their small pages this is lot less less than the AGU ask for (an order of magnitude cheaper than the AGU's free-to-view version), and right at, or even below, the bottom of the range of cost that Jim Giles suggests. I also like their open reviewing system. Now that several of their journals are well-established, it looks like an obvious place to send manuscripts on a wide range of topics. The only thing I really don't much like about their system is that the papers are only available as (awkwardly-formatted IMO) pdfs and not directly as html. But this is a bit of a detail. Sadly, they don't yet have a journal for what I would think of as the bulk of climate science itself (there's clearly a demand for it, as I've seen the occasional paper that I would class as mainstream "Climate Dynamics" material in rather tangentially-related journals like ACP and CP).