Cross-posted from Naturally Selected for added controversy.
Perhaps the most distinctive and powerful thing about Science is its tendency, or rather proclivity to ask searching, even uncomfortable questions. And unlike belief systems, or ideological and political and movements, or pseudoscience, it asks those questions of itself. There’s been a fair bit of that going on recently.
An article in the New Yorker looked at the puzzling phenomenon, as yet unnamed, of seemingly solid observations becoming less reproducible over time. This article received two evaluations on F1000 (here and here), and sparked a lively discussion over at Naturally Selected (= the day job, for those who don’t know).
The New York Times reported on a paper in the 4 January issue of the Annals of Internal Medicine, which claims published clinical trials don’t cite previous trials. Now, there might be trivial explanations for that, but in the same vein Sir Iain Chalmers, editor of the James Lind Library, has some harsh words for scientists. He says there are fundamental and systemic things wrong with the way research, particularly clinical research, is done today.
It’s certainly misconduct
Among these, he accuses (some) researchers of not addressing questions that are
not of interest to patients and clinicians, of failing to contextualize new findings, and being clear about what they’ve actually discovered. He also takes aim at the failure of scientists to publish negative or ‘disappointing’ results. In Ann Intern Med* last year there was a paper, recently evaluated at Faculty of 1000, scrutinizing the reliability of and inherent bias in clinical trials. And today, Nature published a Correspondence arguing that it’s critical to publish negative results.
Interesting times. Are the criticisms Sir Iain makes fair? If so, is this fault of the scientists themselves or the system in which they find themselves working? If the latter, how can it be changed? What about publishing negative results, and reproducibility and publication bias?
Are we questioning Science enough?
*I’m not giving you a link to Ann Intern Med because they don’t do DOIs. Barbarians.