Stimulated, I believe, by Ron Vale’s call to preprints last year, various luminaries from the world of science and science publishing will be gathering in Maryland at the headquarters of the Howard Hughes Medical Institute (HHMI) later this month to discuss the way forward.
The meeting, called Accelerating Science and Publication in Biology – ASAPbio for short – aims to focus discussion on:
preprints and the role that they might play in catalysing scientific discovery, facilitating career advancement, and improving the culture of communication within the biology community.
Admirably the organisers are hoping to get beyond just talking about the well-known problems of scientific publishing:
The meeting will identify actionable next steps that emerge around areas of consensus, and the organizing committee and other interested participants will be involved in subsequent follow-through.
So it’s a serious affair. And the work has already started. In advance of the meeting, Mike Eisen and Leslie Vosshall have uploaded a commentary proposing a mechanism for coupling preprints and post-publication peer review. It’s short and to-the-point and well worth reading. The central feature of their proposed system is that authors would post preprints that could then be peer-reviewed along two different tracks:
Track 1: Organized review in which groups, such as scientific societies or self-assembling sets of researchers, representing fields or areas of interest arrange for the review of papers they believe to be relevant to researchers in their field. They could either directly solicit reviewers or invite members of their group to submit reviews, and would publish the results of these reviews in a standardized format. These groups would be evaluated by a coalition of funding agencies, libraries, universities, and other parties according to a set of commonly agreed upon standards, akin to the screening that is done for traditional journals at PubMed.
Track 2: Individually submitted reviews from anyone who has read the paper. These reviews would use the same format as organized reviews, and would become part of the permanent record of the paper. Ideally, we want everyone who reads a paper carefully to offer their view of its validity, audience, and impact. To ensure that the system is not corrupted, individually submitted reviews would be screened for appropriateness, conflicts of interest, and other problems, and there would be mechanisms to adjudicate complaints about submitted reviews.
Authors would have the ability at any time to respond to reviews and to submit revised versions of their manuscript.
This is an interesting and provocative piece of work but I have some questions that I would like to lob into the discussion.
- Why would scientific societies, many of which have healthy income streams from journal publishing, contribute to a system that would lead to their demise if adopted widely? Could one create incentives for them to participate or should they be sacrificed for the greater good of research?
- Who forms the “coalition” mentioned in the proposal that has the task of approving review groups? This coalition needs to be authoritative for the system to work, but universities are every bit as invested in the current journal system (and JIFs) as researchers. And funding agencies are reluctant to dictate to researchers the routes through which they may publish.
- This new scheme does not guarantee that peer review will occur. Under the present system all competent researchers can get their work reviewed – and be reasonably assured that it will be published. What would tempt them away from journals if the risks of not being reviewed were perceived as tangible?
Journals such as Atmospheric Chemistry and Physics and F1000Research are using more formal types of post-publication peer review – indeed Vitek Tracz and Rebecca Lawrence of F1000 outline its principal features in another ASAPbio commentary (also worth a read). What they offer is a guarantee that review will be conducted and that is something that I think matters to most researchers. I wonder are these new journal formats a more attractive stepping stone away from the present JIF-infected dispensation?
I would be interested to hear other’s responses to both the commentaries and my questions above. It’s great to see meetings like this taking place and I very much hope that a set of actionable points will emerge.
The meeting will be streamed online to enable as many people as possible to follow proceedings, though I hope some of the attendees will take it upon themselves to write pithy summaries.
Related posts:
Pre-prints: Just do it? (Reciprocal Space)
Peer review, pre-prints and the speed of science (The Guardian)
We already have preprint services for biology (https://peerj.com/preprints/ and http://biorxiv.org). We have a robust post publication peer review system, which covers all publications with a DOI, PubPeer.com. We also have many excellent Open Access journals, whose success is due to a combination of the vision of the founders, their quality and the selection pressure on publishing, imposed by funders, primarily, but by no means exclusively in N. America and the UK. Allied to this we have the drive to Open Data, driven also by funders and some scientists.
So why is there a problem that needs solving, by, for example, the ASAPbio proposals?
To quote “The answer lies in the uniquely strong power journals have over careers in biomedicine”.
Views remain entrenched, and not just among the ‘old’ (=my generation), but also among a surprising (to me) number of younger scientists. Moreover, looking beyond the small number of countries currently pushing Open Access, the state of affairs elsewhere is very different, though this is changing.
For me, there are two key players: post publication peer review platforms and open data.
PubPeer.com, for example, provides a platform for post publication peer review for any publication with a DOI and a means for those whose future may depend on the powerful to engage without revealing themselves (anonymity). However, awareness of post publication peer review platforms such as PubPeer.com is still growing. When use of such platforms becomes universal and part of mainstream scientific culture, then many of the problems that have led to the ASAPbio proposals will be resolved. That flawed paper will be just that and it will gather the dust it deserves.
Open data are the other part of the solution, because this provides a means to engage a large number of people in data analysis. A clear advantage is that insights missed by those who acquired the data are likely to surface. This allows in due course an acknowledgement of particular datasets as being groundbreaking, even if the original interpretation was not.
For me the key is that a 21st Century system for dissemination of research must provide an advantage to those who engage with it, while those who remain wedded to 20th Century systems must be at a significant disadvantage. This has to encompass most of the world – and on this front we have a fair way to go. Pushing for greater engagement in existing systems that allow preprints, post publication peer review and open data has to be a priority.
I agree PubPeer.com is a useful addition to the publishing ecosystem but it doesn’t seem to me to address the desire among most authors to have their work reviewed in a timely and systematic manner (admittedly not always achieved through the journal-based system). I also agree, of course, on the point about the overweening influence of journal metrics on careers.
But the real key here, it seems to me, is to figure out realistic and practical steps that can be made to move away from the current system. A revolution may be called for but revolutions require large, dissatisfied populations and, like you, the impression I get from colleagues young and old is that, while they might acknowledge flaws in what happens at present, they derive a certain amount of comfort from the custom and practice that they have known throughout their careers.
Visions for a better future have tremendous power to drive much-needed change – I take my hat off to the likes of Mike Eisen and Stevan Harnad – but that energy needs to be harnessed to a mechanism that the academic community is willing to engage with. That’s not to say that they should simply have a veto on changes; just that a carefully calibrated system of carrot and stick needs to be configured.