A paper was published earlier this week making an extraordinary attack on the integrity of the work of the ENCODE consortium, an international group studying the human genome. Scientists don’t normally go in for this type of public blood letting, making the attack all the more surprising.
Other disciplines are not so reticent. The literary world even hands out a prize, The Hatchet Job of the Year, for the angriest, funniest, most trenchant book review of the past 12 months. This dubious accolade was awarded in February to Camilla Long for her Sunday Times review of Rachel Cusk’s memoir, Aftermath, cataloguing the breakup of her marriage. I remember reading extracts from Cusk’s book and having to give up; I felt as though I was intruding on something that should not have been made public. Long’s review is beautifully written and wonderfully and shockingly harsh referring to Cusk as a “brittle little dominatrix and peerless narcissist”. Here is a longer extract from the review:
“The book is crammed with mad, flowery metaphors and hifalutin creative-writing experiments. There are hectic passages on Greek tragedy and the Christian concept of family, as well as fragments of ghost stories, references to the Anglo-Saxon heptarchy, and heavy Freudian symbolism, including a long description of the removal of a molar, “a large tooth,” she writes portentously, “of great…personal significance”. The final chapter is an out-of-body experience — her situation seen through the eyes of her pill-popping Eastern European au pair”.
Responses from other reviewers were highly polarised: they were either very positive or very negative.
The scientific world rarely behaves like this in public. Not that is to say that we don’t judge our colleagues; we do it all the time. It’s just that we don’t write it down publicly in the same way. Scientific meetings are rife with gossip and I’ve heard character assassinations on more than one occasion at Grant Committees. There is also some trenchant criticism on the blogosphere, as you would expect. Peer-reviewed papers tend to be much more reserved; if we don’t believe another’s work we will say something discrete like “the inability to repeat X’s work is most likely due to methodological differences”. Until recently, the strongest criticism I had seen in a scientific paper was that a competitor’s work was “naïve”.
Earlier this week a paper appeared on line in Genome Biology and Evolution from Dan Graur and colleagues which rewrites these rules. The paper is a critique of work published by the ENCODE consortium last September. To remind you, the ENCODE consortium set out to study that part of the human genome that did not code for proteins. Their headline conclusion, and here I am quoting from the press release, was that 80% of the non coding DNA was functional and comprised millions of switches that regulate the activity of our genes. The press release occasioned many articles in the mainstream media including a silly article by Johnjoe McFadden in the Guardian (Soon science could enable us all to run as fast as Usain Bolt) that spoilt my breakfast one Friday morning. I am not a “DNA/sequencing” person but I objected to the idea of so many, essentially uncharacterised, gene switches and wrote a blog post about this.
At the time, some experts and commentators criticised the ENCODE results in a series of blog posts using strong but polite language. Here are links to Michael Eisen and Brendan Maher both of whom were uncomfortable with the 80% functional figure
Now we have the Graur et al paper with its swingeing critique of ENCODE. Here is the Abstract:
“A recent slew of ENCODE Consortium publications, specifically the article signed by all Consortium members, put forward the idea that more than 80% of the human genome is functional. This claim flies in the face of current estimates according to which the fraction of the genome that is evolutionarily conserved through purifying selection is under 10%. Thus, according to the ENCODE Consortium, a biological function can be maintained indefinitely without selection, which implies that at least 80 − 10 = 70% of the genome is perfectly invulnerable to deleterious mutations, either because no mutation can ever occur in these “functional” regions, or because no mutation in these regions can ever be deleterious. This absurd conclusion was reached through various means, chiefly (1) by employing the seldom used “causal role” definition of biological function and then applying it inconsistently to different biochemical properties, (2) by committing a logical fallacy known as “affirming the consequent,” (3) by failing to appreciate the crucial difference between “junk DNA” and “garbage DNA,” (4) by using analytical methods that yield biased errors and inflate estimates of functionality, (5) by favoring statistical sensitivity over specificity, and (6) by emphasizing statistical significance rather than the magnitude of the effect. Here, we detail the many logical and methodological transgressions involved in assigning functionality to almost every nucleotide in the human genome. The ENCODE results were predicted by one of its authors to necessitate the rewriting of textbooks. We agree, many textbooks dealing with marketing, mass-media hype, and public relations may well have to be rewritten.”
You have to read the text to get the full flavour of the critique but to quote PZ Myers on his blog: “Graur and friends haven’t just poked a hole in the balloon, they’ve set it on fire, pissed on the ashes and dumped them in a cesspit”. A fuller account of the article and its fall out appears on POPSCI.
What are we to make of this? In my opinion, a peer-reviewed paper carries much more weight than a blog post. The paper will be indexed in PubMed and the criticism and language will stand for all to see. Is Graur et al’s criticism of ENCODE justified? I am not an expert but it looks as though it is mostly justified. Is the language of Graur et al justified? Here I am in two minds. On the one hand I enjoy watching a good academic spat and I would expect that, once the anger has cooled, there will be progress in understanding. On the other hand, the derisory and sarcastic language makes me feel uncomfortable and I am very surprised the journal agreed to publish this version of the manuscript. I support the criticism but prefer that this sort of language be left to blogs. More measured language can then be reserved for peer-reviewed papers.
Perhaps we should start a scientific Hatchet Job of the Year award, in which case Graur et al must be favourites to win the 2013 award.
In my experience as an editor, I find that criticisms written in such a trenchant style marginalise the critics (people will think they are mad) and tend to polarise debate rather than resolve it. In my job as an editor with Your Favourite Weekly Scientific Magazine Beginning With N, I have sometimes had cause to ask peer-reviewers to moderate the language of their reports, so as not to be gratuitously offensive, and, particularly, to ask authors to revise their over-hasty rebuttals of reviewers’ reports, so as not to alienate the people whose opinions will influence our decisions on publication. There are those who seem to think that plain speaking in some contexts makes them champions of the First Amendment, but to other people they come over as bullies. (Disclaimer: some of the ENCODE papers were published in YFWSMBWN, though I was not directly involved in their publication.)
Thanks Henry, it’s good to hear the views of an editor. I believe it is always possible to write a strong critique using moderate language and if more extreme language is used it can be very damaging.
It does seem, however, that there was huge anger about ENCODE and that has lead to the Graur et al critique.