I’m a big fan of Mole . And I have to say that wriggling out of reviewing manuscripts is as reprehensible as wriggling out of jury service.
If we take any pride in our vocation (or society), we have to lend a hand in taking out the trash. The doing of science in this particular incarnation depends on faithful peer reviewing. Upholding _ habeus corpus _ is a mainstay of our legal system (I like to think, recent atrocities committed by US and UK governments notwithstanding). Both depend on citizens doing their bit for the social collective. Not only should we do it, we should do it to the best of our ability.
We owe it to the community.
I’m trying to think of how anyone would tackle disagreeing with you.
“No, no. I want to be a selfish bastard.”
I still get asked by senior colleagues to read grant applications. Ostensibly it’s because I’d have a better grasp of English than they do. I usually accept because I get to learn about the cutting edge of another field, so it’s not all bad. But maybe now I should stand up and say, let’s break that chain of laziness! Okay. I am resolved.
Hooray for peer review! If it didn’t exist my job would be more difficult/impossible/ a nightmare. Three hearty cheers!!!
As we’ve discussed on other forums (ask the Editor), I think that a senior person who asks a junior member of the lab to draft the review, and then discusses together, then submits as a jointly authored review to the journal, is doing a good mentoring and educational service. We at nature have discovered many a good new reviewer that way. Agreed that it isn’t fair if the junior person does not get the accreditation.
Slight tangent: what do people (with their peer-reviewer hats on) think of the idea of the peer-reviewers of a submitted paper being able to discuss their reports (once written and sent to the journal) BEFORE the reports are sent on to the author and the author asked to review? Mole in the piece to which you link, Richard, commented about the 50 plus additional experiments he has to do to satisfy peer-reviewers of one of his papers. Would it be a positive move, do you think, if there was a step “reviewer discussion” to allow reviewers to see each other’s reports and perhaps one might concede that every little suggested expt may not be strictly necessary after all? Journals could provide a secure online forum for this purpose — but there are always more suggestions to improve the system than there are resources to do it…..
I don’t think anyone is suggesting that the mentoring-type arrangement you talk about is bad. That probably would take more time than just doing it yourself, actually.
I like the idea in your tangent, Maxine, although I can see how hostile reviewers might get together and comprehensively trash a paper. I’d be happier if initial reviews were deposited, and then a ‘reviewer discussion’ to compare notes. That way shy voices, or dissenting ones (for or against), are not necessarily shouted down.
Oh, Bob: I haven’t been able to come up with a witty riposte to your comment. Suffice it to say, that winning an argument before it gets started is not necessarily all bad 😉
Will a reviewer discussion work in practice? In my experience you usually have one for and one against, which is why you do a tiebreak.
Actually the real problem, given this thread is about doing things properly, will anyone be able to persuade reviewers to spend more time on refereeing (speaking as someone who spent most of the morning refereeing a grant)?
Maxine, my PhD supervisor mentored my first reviewing efforts in almost exactly the way you describe. The experience definitely taught me a lot about the process, which was invaluable when I came to submit my own papers. I don’t think I was named on the review though, but that didn’t seem to hurt me.
Richard: yes, that is what I’m thinking too — the reviewer discussion would take place only after delivery of the written review.
Cath — good to hear that. The point of naming the reviewer from the journal’s perspective is that we ask you directly next time 😉
Cameron — Nature may be somewhat different from some other journals because we are interdisciplinary, but many of our papers are reviewed by two or three scientists with different expertise. I agree science isn’t decided by voting (Gallileo being a good type-example of that); there is some skill in weighing up different perspectives on a piece of work, depending from the discipline of the reviewer.
I think if referees clubbed together and discussed the paper, it could turn people away from their instincts. If one referee has an impression that something is not right, do editors really want the other two to “talk him ’round”? Just as asking a leading question can alter people’s answers, so might committee mentality and peer pressure. Far better, I think, for the editor to retain her job of seeing three raw, uninfluenced opinions and distilling the outcome from the outside.
But then, I am famously old-fashioned when it comes to editing, so take this with a grain of salt.
Is there a halfway house here? Rather than a discussion what would happen if the referees saw each other’s comments? Has anyone done that? It might make people rethink an extremist position?
@Maxine – What does happen when you get those papers where the novelty really lies in the interdisciplinarity, rather than either of the two or three fields that the work draws from? Would having referees see that their opposite number from another field is completely missing the point about ‘their part of the paper promote reassessment?
I take Jennifer’s point, you really don’t want to end up with a committee decision. I still think that the average response of referees to the question of ‘and once you’ve given us a report will you also..’ will be fairly negative.
Cameron and Jennifer — one issue that comes up time and time again is that one of say three referees lists about 20 or 30 points to address that require about a year of work, whereas one of the others suggests one or two points. It is hard for the editor to judge on the technical ground of whether that year of work is reasonable in the context of the paper. The referees already do see each others’ comments after the author resubmits a new ms and a list of point-by-point responses, but the year has happened by then. I am wondering if the reviewers saw each others’ reports before they were sent to the author, whether they might discuss the specific point of how necessary all these experiments are.
This “round” (still at the thought experiment stage!) would be limited to a technical debate on the specifics, it would not be about persuading one referee to amend his or her basic opinion of the work. If the “round” takes place after the initial reports have been provided, the journal has a record of those – the editor would be interested in only this type of discussion between the referees. (In reality, the editor often “over-rules” a referee’s subjective opinion, in either direction: the editor is mainly interested in the referee’s technical judgement.)
Cameron: in response to your interdisciplinary question, I think you are asking two questions.
One type of manuscript is cross-disciplinary, for example might be a large trial requiring a statistician, about a particular disease, requiring someone who is a specialist in that disease. These two referees would be coming from two very different perspectives, and their views can be weighed up pretty easily.
The other is an interdisciplinary paper in itself — for example, biophysics or chemical biology. In those instances, it is harder: as you imply, what is exciting in one discipline can be old hat in another – at Nature, the editors are divided into two teams, physical sciences and biological sciences, and manuscripts are “defined” as being one or the other. However, editors from each team consult each other regularly, for example oceanography from a biological and physical science perspective.
I think you ask a very good question about interdisciplinary research (question type (2) ). Many funding bodies are recognising that it is important, but there isn’t that much of a system for supporting it — in institutions for example. There has been discussion about this in Nature and Nature Network, for example here, here and here.
I should also point out that I haven’t ever refereed for a Nature journal. From what Maxine is saying I think the process and experience looks rather different. I can count on the fingers of one hand the times I have had a paper come back from review for a ‘second round’. I can’t remember in these cases whether I saw the other referee’s original comments or not (although I obviously saw the authors reply to them). I can see it must be tough to arbitrate between (partially) conflicting technical assessments. At one level I am for the, ‘just get it out there and let the conversation run’ camp but that’s because that is the whole agenda I am pushing 🙂
And I’d better not start my rant on mechanisms of support for interdisciplinary research…
Maxine, can I use a specific example re: the paper issue? (feel free to say you can’t comment, but it does have a happy ending). We submitted a paper, originally to Nature three or four years ago. The point of the paper was that we had developed a new approach to calculating DNA melting temperatures. In terms of calculating Tms it wasn’t actually better than the usual approach but it is based on a proper physical model opening up the potential for really probing what is going on, something not possible with an empirical model.
Anyway, the main innovation as we saw it was this extendability that would potentially have real implications for better Tm calculations in the future. So we started off with Nature, went then to N Biotech, and then (I think, this was a while ago so I may have the details wrong) to N Methods. In each case it was returned without review. Now in the end it went into Nature Physics and what was interesting here was that a) we got asked to expand it from a letter to an article and b) it was featured with a N&V. But here the extra interest was driven by the DNA angle, and how it might improve the use of such models in DNA analysis.
Now we are very happy to get a paper into Nature Phys but my feeling is that, while it is interesting to statistical physics people, that if it had been in NB it would have been in front of the biosciences community and may therefore have got more citations – I think the physics community would have picked it up anyway.
I guess the question is how do balance the need for immediate impact (who cares about a new method for calculating Tms?) versus the potential for increasing the impact via the placement of a piece? This is by no means a criticism incidentally – I am interested in how this kind of balance is struck. How important is editorial freedom in making sure that the potentially, but not immediatly obviously, high impact stuff gets through, despite conservative referees? And critically in this day and age, gets through in a timely manner.
And feel free to say we should have written a better cover letter 🙂
Cameron — as you suggest, I can’t comment on particular papers (these are confidential matters to the journal), nor would I wish to as I have not read your work nor am I an expert on it.
But in general terms, the question you ask is exactly the type of question that Nature editors wrestle with. We published the early papers showing direct electron microscopy of DNA, for example, and an early paper showing nanotubes. Sometimes, a new method for achieving a similar result is judged sufficiently novel for publication, and on other occasions, a technique (eg direct visualisation) can be used again to show a conclusion of novel biological interest. Certainly for Nature the criterion of “novel biological interest” is applied to papers considered by the biological sciences team. So a paper demonstrating a better calculation or method of achieving a known result (an improved chemical synthesis pathway, for example) may be more suitable for the physical sciences: and now we do have Nature physics, photonics and nanotech, and soon, chemistry.
I feel that we should really be having this conversation at the Ask the Editor forum (would have a more edifying title for one thing!) where others can see it — but just to note that many very good papers have to be turned away from Nature and the N journals — well over 90 per cent — and also that in the interests of saving authors time, they can submit a presubmission enquiry to assess journal interest before submitting the whole ms – though editors decisions on this cannot be binding, they can provide an indication.
Maxine – I thought that might be the case, I just find it easier to discuss by example 🙂 Also happy to cut and paste across to some other forum to recapitulate the conversation if you feel that is more useful (and/or edifying :).
In some ways this is similar to the discussion you linked to above about what the perceptions are of what is a ‘significant advance’ in the different fields related to an interdisciplinary paper. So whether its nanomedicine or our melting calculations the implications are more clearly seen by the ‘originator’ field than the ‘application’ field yet people from the ‘originator’ field are not well placed to critically analyse the importance or utility of the application.
I would guess that this means that more interdisciplinary work gets published in chemistry, physics, and materials journals than in bioscience journals. Am I wrong on that?
It would be interesting to know what the spread of ‘interdiscplinary’ papers is across the Nature stable and perhaps also how their citation count over time compares to the more traditional papers. You could imagine that the majority might be undercited compared to their comparators whereas a small(ish) number would be sleepers that go ballistic after a delay of some time while everyone else catches up. Of course it would probably depend more than anything else on how you define ‘interdisciplinary’
Maxine,
I was interested by your example of how does an editor chose whether the thirty points by one referee are really justified and the possibility of a discussion between referees. When this happened to me on a submission to an RSC journal (and when I have been a referee myself), the opinion of a third ‘tie-break’ referee is employed to comment independently on this issue. Is this not an approach already used by the Nature editors?
Anna: no, we don’t use “tie break” referees – we assess the content of the comments. Most typically, if the editor is stil of the view that the interest is sufficient for the journal, she or he will ask the author to respond without making a committment to publication. The referees then see how the author responded, in the light of the other referees’ comments. Manuscripts at Nature journals go through several rounds of peer-review, and after the first round, reviewers see all other reviewers’ comments (and are sent the details of the final decision).
We think this is a better approach from the “voting” or “2 against 1” approach.
Richard — what a genteel title to the post 😉
Manuscripts at Nature journals go through several rounds of peer-review, and after the first round, reviewers see all other reviewers’ comments (and are sent the details of the final decision).
Thanks for making that clear, Maxine – having referees see one another’s initial reviews at subsequent rounds of review is often very helpful to the referees themselves, and can act as a corrective against extreme views. Your idea of an additional stage of referee conferring is intriguing, though, and would benefit from an editorial brainstorm, possibly in another place.
Richard—what a genteel title to the post 😉
I think ‘lazy’ is rather rude, actually.