Saturday, July 08, 2006

It's all kicking off on Climate of the Past

Climate of the Past is a fairly new on-line journal with an interesting review/publication model, which I've mentioned before.

This recent hockey-stick-related submission has sparked a flurry of interest, with no fewer than 3 comments (at time of blogging) from Anonymous referee #2, who sets out with "This manuscript is deeply flawed, and its publication would damage the reputation of this promising journal" before moving on to "unpublishability of their submissions", "red herrings which serve only to obfuscate", "spurious claims", "disingenuous cherry pick", "simply impossible to take any of the nonsense they offer up here, at all seriously" etc etc. Ouch. I don't have a dog in this fight and will not dare to offer an opinion :-)

Many of the other manuscripts currently "in open discussion" there are from the special issue (arising from a session at the EGU General Assembly) that Jules is jointly editing. Not that there is any discussion...perhaps it's a shame these aren't collated separately.

27 comments:

Belette said...

Woooo... such excitement. And very promising... now we have http://www.cosis.net/copernicus/EGU/cpd/2/S171/cpd-2-S171.pdf. Pity the refs are anon, and probably a mistake, as I think its clear who at least one of them is.

jules said...

tsk! The special issues are collated! Click on "special issues", 2 lines below "Papes in Open Discussion" on the LHS. Anyway, only 3 papers actually up so far for our one.

ankh said...

I'm sure it'd drive a referee/commentator crazy to try to annotate the cascade of comments and responses -- but I sure wish for such . As the layers of claim and counterclaim and refutation are building up already up, flipping back and forth between PDF files, I can't always tell the cranks from the climatologists.

I trust the real scientists can and will sum up eventually.

EliRabett said...

One guess who anon rev #2 was. Even thought your name is not on these things each of us does have a distinct style. Be that as it may, anon rev #3 blew the paper out of the water.

TCO said...

I find the paper so hard to read stylewise (what is it really about versus what it says it's about) that I haven't had the perseverance to read through and try to understand the issues wrt period splitting, RE with trending series and degrees of freedom. At times, the author makes points which may be likely valid (low degrees of freedom, sample splitting) but does not elaborate with citations or explanations. Given that he is trying to say that basic stats reasoning was flawed in the Mann work, it is incumbent on author to do more than just one liner remarks. He has to prove his point or at least elaborate and give citations. I say this as someone sympathetic to the skeptics and suspicious of Mike's statistical methods.

Given the very poor organization of the paper, I think it would be rejected or at least a major rewrite demanded. The childish review of Anonymous #2 (Mike Mann?) is probably helping the paper more then hurting it, since now people will figure there is something in there and will persevere in reading through the confused explication.

James Annan said...

I haven't even read the paper - I'm just enjoying the comments :-)

I agree that R2's comments would have to be heavily discounted by the editor.

An editor of a similar journal once mentioned that he suspected a weakness of the system was that the first review tended to provoke a series of "me too"s from the other refs. Tactically sound of R2 to get in early then...

TCO said...

Does it sound like it is Mann?

James Annan said...

Oh, I've no idea. I've never met the...er...Mann himself or even read much of what he writes. I guess it must be a strong possibility (and if not him, at least a close ally). Really, it's not my field and I don't find all this linear algebra particularly interesting, but the "debate" makes a good spectator sport, albeit a rather undignified one. I hope my comments on climate sensitivity (and other stuff) are more measured.

Actually, I do have another post coming up which probably relates to his "likely the warmest decade" type of comments, although the origins of my thoughts are in a rather different area.

TCO said...

I thought you were a Mann ally. Hmmm, this is a more complex situation than I realized.

P.s. Could I slip you some Halibucks and get you to run computer programs for Steve? Oh...and write papers for him?

James Annan said...

Heh. I don't really need to go looking for any more enemies right now, especially as I have another broadside up my sleeve awaiting fine tuning (and its a goody, though I say it myself) :-)

But Halibucks are always welcome...

TCO said...

You are one of the people who controls Real Climate, no? How come you act halfway cool and let me talk and stuff? And disagree with some of the Team?

James Annan said...

Oh no, I'm not a member of that particular club. I've written a couple of articles for them, and approve of what they are doing in general terms, but that doesn't mean I automatically endorse every specific detail 100%.

dave frame said...

James wrote: "Really, it's not my field and I don't find all this linear algebra particularly interesting, but the "debate" makes a good spectator sport, albeit a rather undignified one."

The ugliest bits of climate research seem to be those we can't directly check, I reckon. The depth of the Little Ice Age, estimating sensitivity... we're fine when we can do the verification thing, but I think we (as a community) aren't much good at disagreeing about the bits that matter but which we can't use the world as a tie-breaker.

Dave

James Annan said...

The ugliest bits of climate research seem to be those we can't directly check, I reckon.

Interesting characterisation, but I'm not sure I agree really. I'd prefer to say, the disgreements indicate the boundary between what is considered known, and what is not known. Once it is verified, it is known and there is no further room for argument. HvS would probably say something about "post-normal science", but I'm not sure that it isn't just normal science!

dave frame said...

I guess what I mean is we can't put up a satellite and measure the Little Ice Age or the climate sensitivity. Disagreement over the empirically verifiable bits tends to be quite well posed because the bets we make get called in. If the world cup semis and final had never been played I could have argued heroically about why my tenner on Germany was a better bet than (say) Claudio's fiver on France.

Agree with you about "post-normal science", too. Sounds a bit Kuhnian for my liking...

EliRabett said...

I made the point over at Bill's Bar and Grill, but if you are going to rip someone's lungs out it would be a good idea to try and disguise your writing style a bit.

That being said, I think that it would be better in this new open review system to post the solicited reviews simultaneolusly so there would be no piling on.

As to why we need models and reconstructions, it is as idle a thing to wish for data from a satellite launced by William in 1066 to scout out his cross channel vacation as to demand data from 2100. One works with the possible and tries to improve on it.

Finally, and this is my main disagreement with James' approach, is that I think he and his methods overly discounts dangerous outliers.

Phil Hays said...

First two issues have not one single article about a climate warmer than today.

Warmer would be more interesting.

James Annan said...

Phil,

Of course research is most active where there is most data - which generally means recent and therefore usually colder. There is one paper each on MWP and mid-Holocene in the discussion stage, but maybe they aren't quite warm enough for you :-)

The HS-paper comments seem to have dried up for now - but with one more review to come in (and responses), I'll keep an eye on it.

Phil Hays said...

I think that a realistic case can be made that the MWP and mid-Holocene were both cooler than today. MWP based on the English wine index :-)

The drunk looks for his keys under the street lamp. It is not where he lost them, but the light is better there...

Extrapolation is not a good idea.

TCO said...

you want some red meat? Check out the devestating Wegman report and Mann's blustering response on RC. I think Mike is going to lose it. Responsible environmental scientists are going to throw him to the wolves if he doesn't start acting like a truth-seeking scientist instead of a denying child.

BTW, the report actually has a social network chart in it! It shows the "club" that you were talking about. Not a big deal or anything, but pretty funny to see it diagrammed and quantified!

James Annan said...

devestating Wegman report

Gets a big wet raspberry from me. At best it's feeble stuff, and the "social networks" part is positively embarassing. The only "value" in this report seems to have been in generating talking points for the rubes.

TCO said...

How is it feeble that a statistician who is the chair of the NAS Applied Science Panel backed MM up on their MBH crits? If it is because you already accepted the crits fine.

I don't think the SN stuff is bad. I don't think it is a smoking gun either. I think what Wegman did is look at MBH and see how awful it was. That it was not even written in a way that someone could clearly assess the methods. And then he said, how did they get there. And the answer is clearly that Mann did a bunch of wacky stats in a subfield where no one could call him on it.

But now big Daddy is in the house!!!!

James Annan said...

TCO,

I'm not going to go into bat for Mann in any detail (IMO, best to think of MBH as a brave attempt rather than revealed truth) but the Wegman comment misses a crucial point by just looking at the shape of the PC1. Maybe you should try asking M&M about the % variance explained by the PCs in their various "hockey-stick-mining" demonstrations versus that using the real proxies. Talk about non-reporting of adverse results!

ankh said...

>maybe you should try asking

Surely some statistician can be persuaded to write a paper on it, pro bono?

TCO said...

New message from "Tapio" at CP.

ankh said...

> percentage variance....

Oh, is _that_ what people are pointing out the last few days in several places, when they point to two charts that look very alike and note that, when inspected, the scale on one is 10x the scale on the other, so the presentation exaggerates the resemblance of the images?

If so, could you explain why it matters?
Or have you said enough already?

James Annan said...

ankh, I think it's perhaps marginally more subtle than that, but it's along the same lines. The scaling itself is pretty arbitrary. The particular point I am referring to is that the MBH PC1 (1st principal component) explains ~38% of the variance of their data set (from memory), whereas in the M&M random noise test, the PC1 - although somewhat hockey-shaped - only explains about 2% of the variance. IMO this severely weakens their claim that the MBH method generates hockey sticks out of thin air. Note that it's the final result, not the PCs that matter. The PC1 will only dominate the final result if it explains a large part of the variance.