Wednesday, August 16, 2006

More Climate of the Past "Discussion"

With reference to the paper I mentioned last month, I just noticed that the last (presumably) Referee #1 added their comments some time ago...and within 20 minutes, Ref #2 had added a somewhat abusive rejoinder! That's his 4th contribution so far, and he's looking increasingly obsessed IMO. OK, perhaps "abusive" is a bit strong - it is only a rather feeble ad-hom, the sort of thing I wouldn't be surprised to see on usenet or a blog, but hardly what I'd expect in a referee's comment. Maybe I've just had a sheltered life. But anyway, it seems to me that the effort he's making to block the paper is rather more embarassing than the paper's publication itself could ever be.


William M. Connolley said...

Its a bloodbath out there... mind you, its pretty weird to add comments from blogs

Harold Brooks said...

Putting on my journal editor's hat, I'm not particularly excited about the on-line interactive review process from this example.

On the plus side, Reviewer 3 wrote a review that would help me make a decision and, even though the criticisms are substantial, gave guidance to the authors to improve the paper.

I wouldn't be very happy about Reviewer 1 having read Reviewer 2's comments before submitting his or her own review. The review ends up being as much about the other review as it is about the original paper. I like having reviewers having access to the other reviews after they've submitted their own review, especially if, as is typical, there are aspects of the paper that not all the reviewers are expert on. Having an initial review be potentially influenced by another review strikes me as problematic. I could see someone, who had a reasonably good idea who another reviewer was, changing their review to come across as favoring a paper they didn't think was of high quality because of their negative view of the other reviewer. I could also see them ganging up on an author by picking up on a point from another review (even if it is a point outside of their true expertise), making that point seem even more important.

Given the behavior of Reviewer 2 in replying to 1's review and the tone of responses to the responses to the original review, I'd be on the way to disregarding his or her review. I don't think there's a chance that there's anything the authors could do to satisfy Reviewer 2 (short of saying that Mann et al. were exactly right) and I don't think Reviewer 2 will ever move from a reject decision. I'd be at the point where the only reviewer I'd be paying much attention to is Reviewer 3. If the authors can make progress towards satisfying him or her, then we move towards publication. If they show no inclination to do so, stop the process soon and start over.

I'm also not asking Reviewer 2 for any more reviews, probably. He or she seems awfully defensive and not interested in constructive interaction.

James Annan said...

Hi Harold,

I'm pleased to see such eminently sane comments from a journal editor :-) (not greatly surprised, as for the most part editors seem to do a pretty good job IME, but I have had one or two disappointing experiences.)

One editor of one of these open-access on-line journals did mention that he thought that the reviewers tended to pile-on after the first one: a simple solution to that would be to withold all comments (and perhaps even keep the paper itself under wraps) until the official reviews are in, and then publish all the official reviews together. But that would extend the process if there is also a further wait for public comment.