Today sees the publication on bioRxiv of a revised version of our preprint outlining “A simple proposal for the publication of journal citation distributions.” Our proposal, explained in more detail in this earlier post, encourages publishers to mitigate the distorting effects on research assessment of journal impact factors (JIFs) by providing a simple method for publishing the citation distributions that are so incompletely characterized by the JIF.
Since it was first published on 5 July 2016, the preprint has been downloaded well over 11,000 times. It was widely reported in various news outlets and has generated a large volume of commentary on social media (see metrics tab for the article at bioRxiv). As an exercise in post-publication peer review we could hardly have wished for a better response.
In revising our preprint, we have tried to take on board the most substantive criticisms raised online and in follow-up emails from a number of people. These criticisms and our explanation of how we have addressed them are laid out in the Responses to Comments document that is published today alongside the revised preprint (as Supplemental File 4). We are extremely grateful to all those who took the trouble to engage in these discussions and believe that the new version of the preprint provides a much clearer explanation of the rationale behind our work.
We discussed the option of submitting the revised preprint to a peer-reviewed journal but decided in the end not to do go down this route. This decision is primarily motivated by the fact that the preprint has already received extensive peer review from more than a dozen commentators and is unlikely to be altered significantly by further scrutiny. We also feel, given the core message of the article (which is in any case more of a policy paper than a research paper), that there is symbolic value in sticking with a publication venue that does not have an impact factor. However, that choice should not be taken to imply any veiled criticism of the more traditional practices of publication through a scholarly journal, in particular also of work previously posted as a preprint. The mores and modes of academic publishing may currently be the subject of lively discussion but that is a debate for another time and another place.
Finally, we hope that our preprint will continue to be read and discussed, and that its recommendations will be implemented widely by research journals to improve clarity in reporting citation metrics (as some of the journals associated with the undersigned have already done). This is the last revision that we intend to post (barring corrections for any residual errors of fact), so it should be treated as the version of record. Like any traditional journal article, our preprint must now stand or fall on the merits that it has today, the moment of publication.
Vincent Larivière, (Université de Montréal, Canada)
Véronique Kiermer, (PLOS, USA)
Catriona J. MacCallum, (PLOS, UK)
Marcia McNutt, (National Academy of Sciences, USA)
Mark Patterson, (eLife, UK)
Bernd Pulverer, (The EMBO Journal, Germany)
Sowmya Swaminathan, (Nature Research, USA)
Stuart Taylor, (The Royal Society, UK)
Stephen Curry, (Imperial College, UK)
A fascinating article and I hope that you succeed in your campaign to get journals to move from publishing JIFs to a more representative statistic. However, the question that I have for you is does this do anything to encourage the publication of ‘good science’ and to discourage the publication of ‘bad science’. ‘Good science’ needs replication to show that the original experimental results were valid, but unless there is also a change in citing policy to require replication of results to be cited alongside the original results, the replication papers will inevitably receive far fewer citations, with the result that they will be less attractive to journals because they will pull the JIF or other statistics downwards.
There is an interesting article in the latestThe Economist which makes a point about why ‘bad science’ persists: http://www.economist.com/news/science-and-technology/21707513-poor-scientific-methods-may-be-hereditary-incentive-malus?fsrc=scn%2Ffb%2Fte%2lil%2Fed%2Fwhybadsciencepersists . While they use the metaphor of ‘survival of the fittest’ they might equally well have used Gresham’s Law.