{"id":80,"date":"2008-05-26T09:17:26","date_gmt":"2008-05-26T07:17:26","guid":{"rendered":"http:\/\/occamstypewriter.org\/boboh\/2008\/05\/26\/double_blinding_can_we_see_the_point\/"},"modified":"2008-05-26T09:17:26","modified_gmt":"2008-05-26T07:17:26","slug":"double_blinding_can_we_see_the_point","status":"publish","type":"post","link":"https:\/\/occamstypewriter.org\/boboh\/2008\/05\/26\/double_blinding_can_we_see_the_point\/","title":{"rendered":"Double Blinding: can we see the point?"},"content":{"rendered":"

I’ve been asked to write something for the BES<\/a> newsletter on double-blinding for journals. So I thought I would try out this Web2.0 thingy<\/a>, so here’s a draft for you to pick over and comment on. It’s a bit too long, so most suggestions for where to cut will be welcome. Oh, and I couldn’t work out how to have multiple footnotes on the same word, but I hope you can work it out.<\/em><\/p>\n


\nThe traditional single-blinded approach to journal review is in a perpetual state of criticism. One perceived problem is that referees make their assessments based on the identity of the author(s), rather than just on the quality of the paper. This could bias the reviews if the decisions are based on, for example, the gender of the author or on whether the author is well known. An obvious fix for this problem is to double-blind a review: i.e. the reviewers are simply not told the identity of the authors. Sounds great, and a recent survey found that scientists liked the idea, but on further inspection things get more complicated.
\nWe all hear anecdotes about referees holding up papers, or giving bad reviews of papers that disagree with them, but is there more than just anecdote? Firstly, there is at best little evidence for a difference in the quality of1<\/a><\/sup> reviews2<\/a><\/sup>. Secondly, one argument that has been raised for double-blinding is that it removes biases against women authors. But again, there is no evidence for an3<\/a><\/sup> effect4<\/a><\/sup> (this last conclusion has been contended5<\/a><\/sup>, but oddly without contending the analyses and data on which it is based).
\nSo there is little evidence that double-blinding is better for journals. Even worse, the evidence is pretty clear that it does not even work: the proportion of referees that correctly identify the author has been estimated at 322<\/a><\/sup> and 463<\/a><\/sup> percent. Of course, an author is more likely to be recognised if they are known to the reviewer, but this is the same factor that can bias a reviewer in favour of the author. Thus, the two biases may even act together to make the situation more complicated6<\/a><\/sup>.
\nShould journals use double-blind reviewing? There is little evidence that it is beneficial, and the overheads, in terms of ensuring effective blinding, are significant. From the point of view of a journal, the argument in favour is that it might attract better manuscripts: it does not matter if double-blinding has an effect on quality, it only matters that it is perceived to have an effect, and that some authors will chose to submit to a journal because they think they will be more successful. Cynical, I know.<\/p>\n

1<\/sup> van Rooyen, S., Godlee, F., Evans, S., Smith, R., Black, N. (1999). Effect of Blinding and Unmasking on the Quality of Peer Review. J. Gen. Intern. Med. 14:622\u2013624.<\/p>\n

2<\/sup> Justice, A.C., . Cho, M.K., Winker, M.A. Berlin, J.A., Rennie, D. and the PEER Investigators (1999) Does Masking Author Identity Improve Peer Review Quality?: A Randomized Controlled Trial. JAMA. 280: 240-242.<\/p>\n

3<\/sup> Blank, R.M. (1991) The effects of double-blind versus single-blind reviewing: experimental evidence from The American Economic Review. The American Economic Review 81: 1041-1067.<\/p>\n

4<\/sup> Webb, T.J. et al. (2008) Does double-blind review benefit female authors? TREE, in press. DOI: 10.1016\/j.tree.2008.03.003<\/p>\n

5<\/sup> Budden, A.E., Lortie, C.J., Tregenza, T., Aarssen, L., Koricheva, J., Leimu, R. (2008) Response to Webb et al.: Double-blind review: accept with minor revisions. TREE, in press. DOI: 10.1016\/j.tree.2008.04.001<\/p>\n

6<\/sup> As I found out<\/a>. Blank (1991) has evidence that middle-ranked institutions fare worse under double-blind review, which could be used to support my conclusions, but it looks too neat \u2013 I would want to see more studies before declaring success.<\/p>\n","protected":false},"excerpt":{"rendered":"

I’ve been asked to write something for the BES newsletter on double-blinding for journals. So I thought I would try out this Web2.0 thingy, so here’s a draft for you to pick over and comment on. It’s a bit too … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-80","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/posts\/80","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/comments?post=80"}],"version-history":[{"count":0,"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/posts\/80\/revisions"}],"wp:attachment":[{"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/media?parent=80"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/categories?post=80"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/occamstypewriter.org\/boboh\/wp-json\/wp\/v2\/tags?post=80"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}