Earlier this week I had an interesting email from Jesse Shore, President of Australian Science Communicators, about a blog post I wrote back in 2010. Jesse has kindly agreed to let me post our brief correspondence so I’ll let him take up the story.
Dear Professor Stephen Curry,
Although I am belatedly sending you this message (I got side-tracked after I started a draft in 2010), the topic is still very valid.
This regards your article in the Guardian, ‘We’re not on a pedestal: peer review keeps scientists firmly grounded’ (11 Feb 2010), which I appreciated reading. I agree that the peer-review process is powerful and I certainly support it in my dealings with various science sceptics.
Some of these sceptics assert that peer review is a closed shop which excludes the views of scientists which differ from the orthodoxy. This allows them to dismiss the function and value of peer review, especially when it comes to information that is counter to their opinion.
To support their argument they use examples of work requiring a shift of paradigm, where the new interpretation or theory has insufficient evidence to be compelling to convince those holding to the old way of thinking. Sometimes they invoke the story of Barry Marshall and Robin Warren who spent a few years fighting the conventional wisdom that no bacterium could survive the in the human stomach before their work was accepted. I don’t recall how long it took before other research groups repeated their experiments to verify their work and I don’t know whether Marshall and Warren had difficulty in getting published.
This example may involve more than just peer review but there seems to be a link to it. And for the sceptics the rare exception totally undermines the edifice of peer review. Perhaps one can’t easily argue with someone who takes this stance but do you have any good counter arguments for such people?
Cheers,
Jesse
To which I replied:
Dear Jesse
Many thanks for your email.
No-one seriously contends that peer review is flawless but, while it rightfully has its critics, I think the process still serves a very valuable function. It is a means of quality control and its very existence is a testament to the in-built scepticism that is a quintessential part of the scientific enterprise.
Is it a closed shop? I guess it can be seen that way, since only scientific ‘professionals’ are every likely to be asked to participate. Entry can be gained with a PhD but that sort of restriction is no bad thing. In fact it is necessary given the highly specialised nature of most publications these days.
Is the process biased against mavericks and geniuses? Perhaps there’s some risk of that but the stance that ‘extraordinary claims require extraordinary evidence’ is a worthwhile one. For the specific example of Barry Marshall that you quote, although I think there was skepticism of their initial hypothesis, the publication of the clinching experiment (Marshall’s bold self-infection) did not seem to me to be delayed – the experiment was done in 1984 and published in 1985, which seems a pretty standard timeframe. Moreover, it is possible to think of other instances of ground-breaking work where there was little resistance — in large part due to the quality of the accompanying evidence — for example the discovery of RNA interference or the development of PCR.
That’s not to say that there aren’t controversial issues and climate change is a big one. In that case, part of the difficulty lies in the complexity of the data gathering and interpretation. Modelling climate is just really, really difficult so it will be hard for anyone to come up with definitive work. That sort of problem creates space for sceptics who are out to pick holes without properly understanding the scientific process. Don’t know what to do about them apart from keep plugging away and being open about the business of science.
Very best,
Stephen
To which Jesse came back:
Dear Stephen,
Thank you for your considered reply. It is helpful and motivating.
I agree we all have to keep plugging away. One of the challenges is to express the underpinnings of the scientific process and peer review in simple terms. One of my sci-comm colleagues encapsulates them nicely as ‘where’s the evidence?’ and ‘who’s the source?’. Maybe that’s something to build on.
Cheers,
Jesse
I very much agree with him: something to build on for sure.
(Parenthetically – I was also pleased to see that there is life in old blog posts!)
I wonder how influential it was for the wider scientific community that Marshall and Warren were able, once their ideas formed and got some traction, to point to quite a trail of earlier work in the literature that bore on theirs. This stuff had usually been regarded as ‘anomalous’ until the H pylori theory actually explained it, but it did exist. I’m thinking both of spiral bacteria being observed in the stomach, and of things like bismuth salts having efficacy in treating and even curing ulcers. Barry Marshall’s Nobel lecture goes through a lot of this stuff, and is a good listen/read.
The point is that M & W had the key idea, pursued it, did the definitive work and brought it all together – hence the well-deserved Nobel – but there was, it turned out, other relevant older published work. In addition, once M&W’s work had got out in a prominent journal (in about ’85, not all that many years after they started on the H pylori trail), other people got similar results.
One might contrast this with a lot of the so-called ‘paradigm shift’ stuff the anti-science mob tend to drone on about, vaccines and autism being an obvious example.
Thanks for the link to the previous trail of evidence, Austin. It is the key point about the scientific stance which, done properly, has a higher regard for data than theories.
I was struck by this point when catching up with the BBC2 Horizon programme on the Higgs Boson earlier this week (still a few days to watch if you can access iPlayer). For some of the scientists working on the project, many of whom had devoted years to it, it ultimately didn’t matter if they found it or not. Finding the Higgs boson would go a long way to confirming the standard model; but not finding it would be just as exciting, it seemed, since although the theory would have to be discarded, the result would undoubtedly point in new and interesting directions.
It’s an example worth pointing out when faced with the ‘oh, all science is a conspiracy” line.
Being open is indeed the key. Problems occur when people stop being open about their work. For example, people who write things like “Why should I make the data available to you, when your aim is to try and find something wrong with it”.