Monday, January 11, 2010

Open the review process

So the Lindzen and Choi stuff has been effectively trashed (1, 2, 3), and the part GRL has played in the process is not a glorious one.

I'm relieved to see at least that GRL is now starting to be more honest about its attitude towards comments. As you may recall, I've had my problems with them in the past, and I'm glad to see I'm not alone. Perhaps it would be for the best if they bit the bullet and completely abolished comments and replies, which would at least restore some honesty to their behaviour (since in practice they have already abandoned their published policy). However, I don't really like the apparent drift towards comment-in-all-but-name papers which basically focus on errors in a specific previous paper, but which do not give the original authors any opportunity to defend their work against the specific claims. Thus, the disinterested reader may find it hard to judge where the truth lies. There is a hint of this in this Lindzen case, since he has claimed to Andy Revkin that he has a new analysis which sustains his original case (seems unlikely to me in this case, but I mean in general).

A third way could be illustrated by the EGU's approach (eg see here for CP). A major failing in the publication of LC, and probably many similar papers, is the wholly inadequate nature of the peer review they received. Errors cannot be completely eliminated, but I am confident some of the more obvious attempts to game the system would be weeded out - in fact I think most would be dissuaded from even submitting such work, though genuine disagreements can be (and sometimes are) openly aired. Which is, I think, probably useful for the participants, and sometimes amusing for the rest of us :-)

Perhaps the saddest aspect of this story is seeing some apparently talented post-doc try to flush his career down the toilet. Lindzen is a lost cause, Choi is (presumably) not, at least not yet. But if he keeps on putting out shoddy and obviously wrong work, his future may get markedly less bright.

As for the paper itself, it seems hard to defend it as merely honestly mistaken, given the errors identified. However, I haven't seen LC's defence...

6 comments:

crandles said...

comment-in-all-by-name ? but?
However, I haven't seem LC's defence... *seen*

The disinterested reader is just not interested but someone interested without the right background may have problems.

(Says Mr Pedantic Spelling who doesn't like inappropriate spell checker suggestions.)

James Annan said...

Thanks, fixed the typos, but disinterested was what I meant!

EliRabett said...

Well there is always the Motl gambit, go home to a wired position

Steve Bloom said...

Lindzen's last PhD student (in both senses; I don't think he has any more) was made to suffer in the same way. It does seem a little irresponsible on Lindzen's part. OTOH the victims had to have known what they were in for.

Carl C said...

aww, c'mon, do you really think this kid wrecked his career being a postdoc for Lindzen? I know some of his postdocs that are doing quite well. although I guess that's before Lindzen turned into MIT's answer to Fred Singer.

I imagine the poor kid was doing a normal mundane paper on satellite obs or something and Lindzen made him skew it towards his current preferred view? I didn't see anything in Choi's past pubs that was in the "denialist" camp or anything, or published by "Energy & the Environment sponsored by Exxon" etc.

James Annan said...

One bad paper is probably no big deal, but with Lindzen promising a follow-up analysis, he might end up spending a few years making enemies and going down a rabbit hole that he can't easily climb out of.

For a PhD student, I think it is probably harsh to blame them. For a post-doc, I think they should be able to see what they are getting in to - if not, maybe they aren't so bright after all. It is already hard to excuse the errors as honest ones.