As a follow-up to the parenthetical comment I made in a previous post, I thought I would consider the question of IPCC citations more thoroughly. I should start off with a disclaimer, that I consider this sort of bibliometric analysis in general to be a rather limited and blunt tool. For example, I've criticised the h-index in the past, since it rewards co-authorship rather than actual contribution, but on the other hand, it cannot be denied that someone with an h-index of 30 (say) has surely contributed to a large volume of useful research, whereas someone with an h-index of 3 almost certainly has not (yet).
So, I counted up the number of co-authored papers that are cited in the current (first-order) IPCC AR5 drafts. I hope no IPCC authors think this is too naughty, given the semi-private status of these documents, but it's not like I am really giving much away. Also, my totals may be a little bit off, due to the difficulty of identifying "et al" for long author lists, but then again, someone who is nth out of many probably didn't make a huge contribution anyway. I didn't attempt to count the actual citations themselves, because it would have been far too much work, but simply counted up the number of papers that were cited. While of course not all citations represent work of equal importance, it would probably be fair to say that papers which are uncited have not had a great deal of impact, and papers that have not even been written, even less so. Of course, contributing to the IPCC is not necessarily top priority for everyone, but it's a fairly obvious benchmark to indicate the science that people actually care about.
So, I counted up the number of co-authored papers that are cited in the current (first-order) IPCC AR5 drafts. I hope no IPCC authors think this is too naughty, given the semi-private status of these documents, but it's not like I am really giving much away. Also, my totals may be a little bit off, due to the difficulty of identifying "et al" for long author lists, but then again, someone who is nth out of many probably didn't make a huge contribution anyway. I didn't attempt to count the actual citations themselves, because it would have been far too much work, but simply counted up the number of papers that were cited. While of course not all citations represent work of equal importance, it would probably be fair to say that papers which are uncited have not had a great deal of impact, and papers that have not even been written, even less so. Of course, contributing to the IPCC is not necessarily top priority for everyone, but it's a fairly obvious benchmark to indicate the science that people actually care about.
| Author | # Papers cited / 1st author |
| Hargreaves | 16 / 3 |
| IPCC LAs | |
| Abe-Ouchi | 17 / 1 |
| Kitoh | 14 / 5 |
| Emori | 9 / 1 |
| Kimoto | 8 / 0 |
| Nakajima | 6 / 1 |
| Takemura | 5 / 2 |
| Kondo | 4 / 1 |
| Aoki | 3 / 2 |
| RIGC TLs | |
| TL1 | 9 / 0 |
| TL2 | 5 / 4 |
| TL3 | 3 / 0 |
| TL4 | 2 / 2 |
| TL5 | 2 / 0 |
| TL6 | 2 / 0 |
| ... | 0 or 1 |
According to my counting, jules has co-authored 16 cited papers (I missed one before) and actually wrote 3 of them herself. Below her, I've listed equivalent values for the Japanese IPCC Lead Authors. These include several of the most productive and eminent active climate scientists in the country. (There is a another whole cadre of emeritus and super-senior types above them, but they don't actually write much, and it's mostly this "younger" generation - where "younger" is 40s-50s - who are leading the research.) Anyone working in climate science is likely to know of most of the top few of this group, but the bottom ones...well, even after 10 years here I had to look them up. To be fair, these guys are mostly genuinely young(er than me :-)) and are presumably considered up-and-coming rather than leading figures.
I also checked on most of the Team Leaders in RIGC - these are the ones whose management positions are effectively protected in a way that jules was not, according to the arbitrary decree of JAMSTEC - and have put a bunch of the best scores I obtained in the table. I don't want to embarrass these guys by identifing them here publicly (though they are all available on this page), I think that at least some of them do good and interesting work and have no real complaint with them holding their positions. The top person on this section has a full-time faculty position at Tokyo University as well as a bit of an army at RIGC, but I don't need to bother with too much special pleading to explain away his score, as despite the huge disparity in resources and position he only scraped to just over half as many papers cited as jules does. I suppose I could have potentially included Abe-san and Emori-san in this section of the list too, due to their RIGC positions - as I have mentioned before, JAMSTEC's short-term contract ideology lead to a highly unsatisfactory absentee management culture that still remains here to a significant degree. I didn't bother to list individually the many TLs who score only 1 or less. Some of these may have the excuse that their main focus just isn't that IPCC-relevant, but that certainly won't wash for all of them, or at least it shouldn't - the clue is in the name, "Research Institute for Global Change"! The fact that the TL results are rather lower than those of the IPCC Lead Authors, seems to broadly support my use of the metric as a rough guide to impact.
Some of our colleagues and collaborators in non-management positions are quite productive, and have several cited papers - for those I looked up, their scores are anything from 1-9 in total papers, with up to 5 as first author for the best of the bunch. Interestingly, the highest values here are all for people outside of RIGC. It's not that RIGC people are particularly stupid or lazy, but for the most part there is an absence of any meaningful mentoring or leadership, such that many people just waste their time going down irrelevant rabbit-holes that no-one outside of their cubicle walls actually cares about. FWIW, my values are 13/4, but as a sort of part-manager in collaboration with jules, that comparison may be a little flattering.
As a reality check, I also looked up a bunch of people who I expected to have made a really significant impact - in no particular order, Gavin Schmidt, Mat Collins, Mike Mann, Hugues Goosse. And sure enough, they all have more (or many more) papers cited, as befits the substantial contributions they have made across a wide range of topics. I haven't actually calculated their numbers as I got bored of counting (and checking duplicates) at about 20! If you haven't heard of Hugues, he was the EGU's young scientist of the year back in 2005 and has remained amazingly productive since then. Mat Collins was Head of something-or-other at the Hadley Centre before moving to a professorship at Exeter Uni. The other two certainly need no introduction. So I don't want anyone to think we are claiming to be really important on the cosmic (or even global) scale. But in the context of what more normal people seem to have achieved, certainly within the small fishpond that is Japan, or the tiny puddle that is RIGC, I think it would be a challenge to argue that our contribution has been so piss-poor as to actually merit demotion and a disbanding of the research group. Nevertheless, this is JAMSTEC, and that is what they have done.
14 comments:
IEHO first author is an often vastly overrated statistic. The mathematicians are at least honest and use alphabetically order. Still you have come up with an interesting idea
Oh, I think 1st author is worth something - IME this person has almost always done the largest proportion of the work, except perhaps in the massive "MIP"-type collaborations - and even then, they take responsibility for bringing things together.
I do think the maths system is fine for maths. And I could live with alphabetical myself. But I would be tempted to change my name to Aanan, just to make sure :-)
Well, you seem to be moving in a well worn Gaijin direction. Japan is about Harmony. Seriously.
It's not about "treading on the toes of giants".
Isn't it standard bureaucratic procedure to protect the questionable part of the budget (or workers) and leave the option to cut the part most recognized as essential?
Shibui:
"Japan is about Harmony"
...and suicide.
Yes but, Jules, you are English!
Don't forget the lines written by Rudyard Kipling:
"If you can meet with Triumph and Disaster and treat those two impostors just the same"
HTH,
Cheers, Alastair.
She'll be a man?
That poem couldn't be written now in politically correct Britain. The last line would have to read:
"And - which is more - you'll be a Woman/Man, my daughter/son!"
And that doesn't scan :-(
I am fascinated that you have no concern about repercussions from posting this stuff on your blog. It all seems rather unJapanese to be implying anything other than your work and bosses are excellent and you are not deserving of such a rich and productive position
I was wondering that. Is James hoping that his bosses won't read this, or is he hoping they will?
Presumably, if they did read it, face would require them to pretend not to have.
My boss had already read, edited and approved this message before it was posted, but then again, she has just been sacked for her poor performance :-)
Some of you may not realise how normal it is for scientists - at least some of us - to speak bluntly and openly. In fact I have presented this table of results to my "team leader" (not jules) and was slightly less restrained in my language than what I wrote here. I will probably show it to a wider audience next time I give an informal seminar - and as seminar organiser, that may be soon!
James, I have enjoyed reading some of your papers and also those by Dr. Hargreaves. While we probably disagree on other things relating to funding priorities I agree completely that it is ludicrous to regard the salaries paid to researchers like yourselves as something that can be cut.
Alex, thanks for the thought, luckily you don't know what we are paid :-)
Post a Comment