Bad review, good review, peer review, Parliamentary review

Are neutrinos faster than light?

This is a big question at the moment since they have been ‘observed to be’ by a group of researchers recently. I myself am sceptical, Jon Butterworth wrote an excellent blogpost about this which points out some of the problems. However if it were to be true, its a REALLY BIG DEAL.

In Twitterland the press announcement of super-fast neutrinos PRE peer review was somewhat derided by some; where they thought this announcement should have gone through a peer review process before the news was ever released.

On balance, I don’t think so, this is an example of a potential REALLY BIG RESULT (see here for a good physics lite description ) or a potential REALLY BIG MISTAKE – which is what happened with cold fusion, if you remember that story. If you don’t the story is basically, scientists see really big result, scientists don’t double test the result, scientists claim they see cold fusion, other scientists can’t reproduce their results, original scientists refuse to believe they were wrong (pathological); but (at least to date) they still can’t produce enough energy to even warm a cup of coffee – there is a nice review about cold fusion and its subsequent debunking here.

The cold fusion folk in Utah also didn’t go to peer review before the press release, but if their result had been correct, well, we would think differently about it now. A result like this would be SO big that you want other people to know about it as soon as you can. Most day-to-day, even big scientific breakthroughs don’t produce a really big result like this, not in the sense of overturning the last 100 years of physics, quantum chemistry and the like. I can see the need to release the news quickly, not to gloat about your success but rather to tell the rest of the scientific community so they can verify or not verify, and peer review can take a long long time.

And we also know the peer review process has room for improvement, how? UK Parliament says so. If you recall, a few months back a UK Parliamentary committee sat to review and discuss the peer review process in journals and they concluded (according to the BBC ) that, wait for it,

there is room for improvement, data should be publicly available and that there should be formal training for reviewers. Even though the UK parliament has no authority over international science journals, none. Is the American Chemical Society really going to care what UK Parliament says about their peer-review practices? No.

I am a big defender of peer review, but anyone is always free to publish their data on the web if they want, in fact some people only ever do that. However, at least in my field, if you want to maintain any kind of reasonable reputation among your colleagues and peers as a research scientist, you have to have peer reviewed publications.

If super speedy neutrinos are real – its such a potentially big result that a press release allows for verification (or nullification) to happen faster. THEN comes peer review, for everyone who works on these systems. In order to be ‘vetted’ scientifically this science will almost certainly go through a peer review process. Being a big fan of peer review, and sarcasm aside there is always room for improvement with this process, especially as it effects the publication of new ideas and where fraudulent science ekes through, but your work should be evaluated by your peers.

To close on a slightly tangential note, peer review seems to get a lot of crap in the blogosphere (and by Parliament) however it does often work quite nicely. I just had a paper accepted (barring minor revisions); it was reviewed by four referees; over 2 rounds of review. One of the referees, there is always one, was very negative, however I think negative reviews are sometimes better than good reviews even, as they make you have to THINK about what you are doing. Maybe the referee didn’t understand the paper? Which means you need to rewrite it to be more clear. Maybe the referee hates your technique? Which means that you have to justify the use of your technique. Maybe the referee thinks your result is wrong? Then you have to provide further evidence. These are all good things.

About Sylvia McLain

Girl, Interrupting aka Dr. Sylvia McLain used to be an academic, but now is trying to figure out what's next. She is also a proto-science writer, armchair philosopher, amateur plumber and wanna-be film-critic. You can follow her on Twitter @DrSylviaMcLain and Instagram @sylviaellenmclain
This entry was posted in fast neutrinos, Peer review, Uncategorized and tagged , , . Bookmark the permalink.

2 Responses to Bad review, good review, peer review, Parliamentary review

  1. Frank says:

    I agree that Select Committee enquiries can sometimes seem a bit irrelevant, or too late to affect an outcome, but they do provide a useful way to round up the evidence on a topic and can sometimes help to move the debate forward.  In this case, they have affirmed the importance of pre-publication peer review, whilst noting that “different types of peer review are suitable to different disciplines and research communities”.

    I’m not an expert on the publishing culture of physics, but I had understood that in high-energy physics it was common (even before the Arxiv) for preprints to be widely shared. Hence it is pretty unremarkable to find a paper on Arxiv that has not yet been peer-reviewed. The press release stated “we need to be sure that there are no other, more mundane, explanations. That will require independent measurements”. That sounds like a humble acknowledgement of surprise and puzzlement at their findings to me, rather than a hubristic fanfare akin to Pons and Fleischmann’s cold fusion “discovery”.  

  2. I agree that the scientists were much more sceptical themselves, I thought their press release was more than reasonable, I should have made that more clear in my post. Thanks for bringing that up.

Comments are closed.