Earlier this week I had an interesting email from Jesse Shore, President of Australian Science Communicators, about a blog post I wrote back in 2010. Jesse has kindly agreed to let me post our brief correspondence so I’ll let him take up the story.
Dear Professor Stephen Curry,
Although I am belatedly sending you this message (I got side-tracked after I started a draft in 2010), the topic is still very valid.
This regards your article in the Guardian, ‘We’re not on a pedestal: peer review keeps scientists firmly grounded’ (11 Feb 2010), which I appreciated reading. I agree that the peer-review process is powerful and I certainly support it in my dealings with various science sceptics.
Some of these sceptics assert that peer review is a closed shop which excludes the views of scientists which differ from the orthodoxy. This allows them to dismiss the function and value of peer review, especially when it comes to information that is counter to their opinion.
To support their argument they use examples of work requiring a shift of paradigm, where the new interpretation or theory has insufficient evidence to be compelling to convince those holding to the old way of thinking. Sometimes they invoke the story of Barry Marshall and Robin Warren who spent a few years fighting the conventional wisdom that no bacterium could survive the in the human stomach before their work was accepted. I don’t recall how long it took before other research groups repeated their experiments to verify their work and I don’t know whether Marshall and Warren had difficulty in getting published.
This example may involve more than just peer review but there seems to be a link to it. And for the sceptics the rare exception totally undermines the edifice of peer review. Perhaps one can’t easily argue with someone who takes this stance but do you have any good counter arguments for such people?
To which I replied:
Many thanks for your email.
No-one seriously contends that peer review is flawless but, while it rightfully has its critics, I think the process still serves a very valuable function. It is a means of quality control and its very existence is a testament to the in-built scepticism that is a quintessential part of the scientific enterprise.
Is it a closed shop? I guess it can be seen that way, since only scientific ‘professionals’ are every likely to be asked to participate. Entry can be gained with a PhD but that sort of restriction is no bad thing. In fact it is necessary given the highly specialised nature of most publications these days.
Is the process biased against mavericks and geniuses? Perhaps there’s some risk of that but the stance that ‘extraordinary claims require extraordinary evidence’ is a worthwhile one. For the specific example of Barry Marshall that you quote, although I think there was skepticism of their initial hypothesis, the publication of the clinching experiment (Marshall’s bold self-infection) did not seem to me to be delayed – the experiment was done in 1984 and published in 1985, which seems a pretty standard timeframe. Moreover, it is possible to think of other instances of ground-breaking work where there was little resistance — in large part due to the quality of the accompanying evidence — for example the discovery of RNA interference or the development of PCR.
That’s not to say that there aren’t controversial issues and climate change is a big one. In that case, part of the difficulty lies in the complexity of the data gathering and interpretation. Modelling climate is just really, really difficult so it will be hard for anyone to come up with definitive work. That sort of problem creates space for sceptics who are out to pick holes without properly understanding the scientific process. Don’t know what to do about them apart from keep plugging away and being open about the business of science.
To which Jesse came back:
Thank you for your considered reply. It is helpful and motivating.
I agree we all have to keep plugging away. One of the challenges is to express the underpinnings of the scientific process and peer review in simple terms. One of my sci-comm colleagues encapsulates them nicely as ‘where’s the evidence?’ and ‘who’s the source?’. Maybe that’s something to build on.
I very much agree with him: something to build on for sure.
(Parenthetically – I was also pleased to see that there is life in old blog posts!)