Quite a bit of attention has been focused on article retractions since Ivan Oransky launched his Retraction Watch blog last year. One recent discussion in blogworld looked at when a retraction is warranted and what a retraction means. Dr Isis suggested that any retraction carries a whiff of fraud, and thus this measure should be used sparingly. Her take-home message, though, is:
it is important to review the related literature in its totality. I am told, after all, that this skill is a critical character of the PhD.
I like that phrase “literature in its totality”. It is a useful reminder that quick-and-dirty searching is not always the answer; sometimes a thorough literature search is useful. This is what librarians are trained to do.
This theme is echoed by Bonnie Swoger, a US librarian blogger, who has highlighted some recent research on retractions and rebuttals which suggests that scientists “aren’t always finding, reading or critically analyzing the original and rebuttal papers”.
A study in the ecology field looked at seven high profile papers originally published in Science or Nature, all of which had at least one rebuttal published. Papers that cited either the original article or the rebuttal were identified and their citations examined. It was found that:
- Original papers were cited 17 times more than the rebuttals.
- Several papers cited only the original paper, and 95% of these accepted the original at face value
- Only about 5% of the citations to the original papers were critical (at all) of the original article.
- Some papers cited the original and the rebuttals as though they both supported the same position!
The authors suggest that original articles and rebuttals need to be better linked in information retrieval systems. Bonnie Swoger suggests that researchers should put a bit more effort into their searches. Perhaps asking the library for searching assistance would help too.
The study looked only at papers in the field of fisheries policy, and the comments on the blog post suggest that its authors have a particular point of view that is motivating their ‘complaint’ about non-citation of rebuttals. It may be that in biomedical fields the situation is better because PubMed does try to link between original articles and retractions or comments.
NCBI’s published policy on tracking retractions, corrections etc suggests that:
Users who search MEDLINE will be informed if they retrieve a citation for an article that has been corrected by an erratum notice, retracted or partially retracted, corrected and republished, been found to duplicate another article, generated a separately published commenting article, been updated by a subsequent article, if a summary for patients has been published, or has been republished (reprinted) in another journal.
A recent example is the 2009 XMRV paper in Science. The Pubmed record links into various comments and to the recent editorial expression of concern about the paper. However, the PubMed record for another controversial article, a much-criticised GWAS study, has no link to an editorial expression of concern that was issued. I haven’t studied the efficiency of PubMed’s links to commentaries and retractions, so can’t say if this was a one-off omission or a common failure. PubMed of course only refers to published journal articles, not to critiques published on blogs.
A 2008 paper in the Journal of Medical Ethics suggests retractions are being missed by researchers in the biomedical field. It found that 315 retracted papers from English journals were cited 3942 times before retraction and 4501 times after retraction:
Although retractions are on average occurring sooner after publication than in the past, citation analysis shows that they are not being recognised by subsequent users of the work. Findings suggest that editors and institutional officials are taking more responsibility for correcting the scientific record but that reasons published in the retraction notice are not always reliable. More aggressive means of notification to the scientific community appear to be necessary.
During a brief Twitter exchange Cameron Neylon, with characteristic Antipodean understatement, suggested an alternative system of notification.
Until then, the only way to find retractions etc is to look for them, or ask someone else to look for them – such as your librarian.
Original paper:
Banobi, J., Branch, T., & Hilborn, R. (2011). Do rebuttals affect future science? Ecosphere, 2 (3) DOI: 10.1890/ES10-00142.1
This is going to sound naive, but would a librarian really help a researcher find citations for his/her paper? The thought had never occurred to me, and I’m sure it’s not occurred to most of my colleagues.
Jenny – Until about 15 years ago that is what I spent most of my working day doing. Spending a lot of time searching means you become familiar with various little tricks and vagaries of search services, and with a range of different search tools rather than always sticking to one method. Hence I could find articles that our researchers didnt find for themselves.
But, as PubMed etc came along, the advantage of being able to search for yourself just when you want to began to outweigh the advantage of having that outside help. Nowadays quick-and-dirty is the norm and people rarely request systematic literature searches.
Curiously in clinical medicine it is different and librarians are at the heart of literature searching and critical analysis for systematic reviews in the evidence-based medicine field.
I like Cameron’s suggestion very much! In fact, as I was reading the first few paragraphs of your post, I was thinking “a big flashing light and siren on all PubMed and journal website abstracts of retracted papers might work…”
Well, I think its worth a trial somewhere to see if it makes any difference!