As a follow-up to the parenthetical comment I made in a
previous post, I thought I would consider the question of IPCC citations more thoroughly. I should start off with a disclaimer, that I consider this sort of bibliometric analysis in general to be a rather limited and blunt tool. For example, I've
criticised the h-index in the past, since it rewards co-authorship rather than actual contribution, but on the other hand, it cannot be denied that someone with an h-index of 30 (say) has surely contributed to a large volume of useful research, whereas someone with an h-index of 3 almost certainly has not (yet).
So, I counted up the number of co-authored papers that are cited in the current (first-order) IPCC AR5 drafts. I hope no IPCC authors think this is too naughty, given the semi-private status of these documents, but it's not like I am really giving much away. Also, my totals may be a little bit off, due to the difficulty of identifying "et al" for long author lists, but then again, someone who is nth out of many probably didn't make a huge contribution anyway. I didn't attempt to count the actual citations themselves, because it would have been far too much work, but simply counted up the number of papers that were cited. While of course not all citations represent work of equal importance, it would probably be fair to say that papers which are uncited have not had a great deal of impact, and papers that have not even been written, even less so. Of course, contributing to the IPCC is not necessarily top priority for everyone, but it's a fairly obvious benchmark to indicate the science that people actually care about.
Author
| # Papers cited / 1st author
|
Hargreaves
| 16 / 3
|
IPCC LAs
|
|
Abe-Ouchi
| 17 / 1
|
Kitoh
| 14 / 5
|
Emori
| 9 / 1
|
Kimoto
| 8 / 0
|
Nakajima
| 6 / 1
|
Takemura
| 5 / 2
|
Kondo
| 4 / 1
|
Aoki
| 3 / 2
|
RIGC TLs
|
|
TL1
| 9 / 0
|
TL2
| 5 / 4
|
TL3
| 3 / 0
|
TL4
| 2 / 2
|
TL5
| 2 / 0
|
TL6
| 2 / 0
|
...
| 0 or 1
|
According to my counting, jules has co-authored 16 cited papers (I missed one before) and actually wrote 3 of them herself. Below her, I've listed equivalent values for the Japanese IPCC Lead Authors. These include several of the most productive and eminent active climate scientists in the country. (There is a another whole cadre of emeritus and super-senior types above them, but they don't actually write much, and it's mostly this "younger" generation - where "younger" is 40s-50s - who are leading the research.) Anyone working in climate science is likely to know of most of the top few of this group, but the bottom ones...well, even after 10 years here I had to look them up. To be fair, these guys are mostly genuinely young(er than me :-)) and are presumably considered up-and-coming rather than leading figures.
I also checked on most of the Team Leaders in RIGC - these are the ones whose management positions are effectively protected in a way that jules was not, according to the arbitrary decree of JAMSTEC - and have put a bunch of the best scores I obtained in the table. I don't want to embarrass these guys by identifing them here publicly (though they are all available on
this page), I think that at least some of them do good and interesting work and have no real complaint with them holding their positions. The top person on this section has a full-time faculty position at Tokyo University as well as a bit of an army at RIGC, but I don't need to bother with too much special pleading to explain away his score, as despite the huge disparity in resources and position he only scraped to just over half as many papers cited as jules does. I suppose I could have potentially included Abe-san and Emori-san in this section of the list too, due to their RIGC positions - as I have mentioned before, JAMSTEC's short-term contract ideology lead to a highly unsatisfactory absentee management culture that still remains here to a significant degree. I didn't bother to list individually the many TLs who score only 1 or less. Some of these may have the excuse that their main focus just isn't that IPCC-relevant, but that certainly won't wash for all of them, or at least it shouldn't - the clue is in the name, "Research Institute for Global Change"! The fact that the TL results are rather lower than those of the IPCC Lead Authors, seems to broadly support my use of the metric as a rough guide to impact.
Some of our colleagues and collaborators in non-management positions are quite productive, and have several cited papers - for those I looked up, their scores are anything from 1-9 in total papers, with up to 5 as first author for the best of the bunch. Interestingly, the highest values here are all for people outside of RIGC. It's not that RIGC people are particularly stupid or lazy, but for the most part there is an absence of any meaningful mentoring or leadership, such that many people just waste their time going down irrelevant rabbit-holes that no-one outside of their cubicle walls actually cares about. FWIW, my values are 13/4, but as a sort of part-manager in collaboration with jules, that comparison may be a little flattering.
As a reality check, I also looked up a bunch of people who I expected to have made a really significant impact - in no particular order, Gavin Schmidt, Mat Collins, Mike Mann, Hugues Goosse. And sure enough, they all have more (or many more) papers cited, as befits the substantial contributions they have made across a wide range of topics. I haven't actually calculated their numbers as I got bored of counting (and checking duplicates) at about 20! If you haven't heard of Hugues, he was the EGU's young scientist of the year back in 2005 and has remained amazingly productive since then. Mat Collins was Head of something-or-other at the Hadley Centre before moving to a professorship at Exeter Uni. The other two certainly need no introduction. So I don't want anyone to think we are claiming to be really important on the cosmic (or even global) scale. But in the context of what more normal people seem to have achieved, certainly within the small fishpond that is Japan, or the tiny puddle that is RIGC, I think it would be a challenge to argue that our contribution has been so piss-poor as to actually merit demotion and a disbanding of the research group. Nevertheless, this is JAMSTEC, and that is what they have done.