Wednesday, August 21, 2013

Geoscientific Model Development: the world's best journal just got better!


Disclaimer: although I'm one of the founding executive editors, I'm not speaking on their behalf.

GMD was set up about 5 years ago "to promote model development as a serious and worthwhile activity, by providing a home for papers covering a wide range of aspects of the subject." I previously blogged a little about it here on the occasion of it first being included in the ISI list.

We recently got our new impact factor, which now is a shade over 5 - an impressive value that places the journal a whisker behind the front-runner ACP in the EGU stable, and in the top 20 of all of the vast array of relevant geoscience-type journals (according to another ed who looked at the list - it seems broken at the moment). For context, GRL and Journal of Climate are about 4, the JGR family around 3. Basically, little beyond the tabloids and some review journals are more highly cited. Submissions are also still rising strongly. So clearly we were right, there was indeed a gap in the market and by any reasonable measure GMD has been wildly successful.

So we've changed it.

It had always been our hope that papers describing new models would be accompanied by the actual code. This would ensure persistence and traceability of models, and hopefully help to propagate good practice. But as a new journal (and one that was establishing an entirely novel niche) we didn't think we were in a position to require this. And while it was always encouraged, this wasn't enough in practice - only a very small number of authors actually provided the code. Now we are much better established and successful, and have decided it's time to take this step:
The paper must be accompanied by the code, or means of accessing the code, for the purpose of peer-review.
Just to be clear, the reviewers are not required to review the code - this in some cases will be wholly impractical. Some models are massive, and/or tied to specific computer architectures. But the principle is clear. I'm hopeful that this requirement, together with a new mandatory section on wider code availability, should help the push towards open access.

There are various other more minor modifications (eg rationalisation of manuscript types) which we have made in light of what is now several years of experience. The full editorial is here, which also includes a link to the original proposal, as supplementary info.

7 comments:

jules said...

disclaimer: impact factors can go down as well as up...

ob said...

Do you expect the changes to impact the number of submissions?

James Annan said...

I doubt it will have that much effect. I suppose it will seem like an unwelcome burden to some, but it will enhance those papers that do fulfil the conditions. A couple of years ago we wondered if the imposition of page charges would hit submissions (the first year or two, publication was free) but they are still rising strongly.

Nick Barnes said...

This is fantastic news. Very pleased.

Francis Turner said...

It looks to me like this section of A1 (and the related bit in the abstract) somewhat defeats the whole point:

"When copyright or licensing restrictions prevent the
public release of model code, or in the cases when authors do not wish to allow access to the code, editors and
referees must be given access to the model code for the
purpose of the review (whilst preserving the anonymity
of the referees)."

It seems to me you need to insist on some reason other than "do not want to" for why the source code and/or compiled object code would not be made available. It is obviously not not required to include the basic compilers, language modules, libraries etc. because a) assuming the versions required are properly documented there's no point and b) many such tools (e.g. Matlab) are commercial offerings so they can't be distributed.

However in such cases the expectation should surely to release the source code of the model. If it is sufficiently clever that the authors want to patent it/sell it etc. then maybe you insist that they file the patent and/or set up the sales organization before publication.

In other words I think you need to have some kind of disincentive to not making the code publicly available.

At the very least I think you should have some clear statement that lack of public availability means a lower likelihood of publication and/or that lack of public availability means an extra charge because the journal will pay someone to reproduce the model and results and confirm that it performs as claimed and maybe also do some sort of code review to verify code quality.

James Annan said...

Well, we certainly aren't in the business of paying anyone anything related to code verification - in common with all journals, the reviewing is on a purely voluntary basis (though the underlying publishing company has paid staff).

We hope that the requirement to make the code available to reviewers, plus the statement of reasons for not releasing it more widely, will encourage a large proportion to just make it open. After all, pretty much everyone wants their code to be widely used anyway (= citations, fame and fortune). Of course, the rules may be modified in light of experience. No-one has done anything like this before, and we want to bring the authors with us as we develop the publication model, rather than scare them off with stiff demands.

Francis Turner said...

I hope you are right and peer pressure/bragging rights etc. work well enough