{"id":1497,"date":"2012-04-22T16:15:14","date_gmt":"2012-04-22T16:15:14","guid":{"rendered":"http:\/\/occamstypewriter.org\/scurry\/?p=1497"},"modified":"2012-07-27T08:29:21","modified_gmt":"2012-07-27T08:29:21","slug":"eyes-on-the-prize-are-blind-to-reality","status":"publish","type":"post","link":"https:\/\/occamstypewriter.org\/scurry\/2012\/04\/22\/eyes-on-the-prize-are-blind-to-reality\/","title":{"rendered":"Eyes on the prize are blind to reality"},"content":{"rendered":"<p><em>Scientists&#8217; quest for publication in journals with high impact factors is widely perceived as one of the more refractory barriers to the fuller adoption of open access, which I believe to be in the <a href=\"http:\/\/www.guardian.co.uk\/commentisfree\/2012\/apr\/10\/science-open-access-publishing\">best interests of science<\/a>. <\/em><\/p>\n<p><em>But the barrier problem is <a href=\"http:\/\/blog.mysciencework.com\/en\/2012\/04\/17\/advances-in-open-access-a-la-nature-publishing-group.html\">complicated<\/a>. Some of its dimensions were teased out in <\/em><em>a debate on &#8216;Open Science and the Future of Publishing&#8217; held at Oxford at the end of February this year that involved publishers, funders and proponents of open access. The <a href=\"http:\/\/www.youtube.com\/watch?v=yELZ3kbFj1w\">video<\/a> of the debate is well worth watching (especially the first 45 min &#8211; though there&#8217;s a <a href=\"http:\/\/f1000research.com\/2012\/03\/02\/open-science-and-the-future-of-publishing-a-round-up-of-this-weeks-debate\/\">nice summary<\/a> over at F1000). I was particularly struck by the comments made by Alison Mitchell of the Nature Publishing Group (NPG) (from 10:50 &#8211; 19:12) which started a train of thought about their gold-standard publication, Nature. See if you think I&#8217;m going off the rails.<\/em><\/p>\n<p><iframe loading=\"lazy\" title=\"Evolution of Science: Open Science and the Future of Publishing\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/yELZ3kbFj1w?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p><strong>The problem<\/strong><\/p>\n<p>Publication in <a href=\"http:\/\/www.nature.com\/nature\/index.html\">Nature<\/a> is a highly coveted prize. Many labs will literally pop champagne corks if their paper is accepted by one of the most <a href=\"http:\/\/en.wikipedia.org\/wiki\/Nature_(journal)\">prestigious scientific journals<\/a> in the world. I shared in this wonderful feeling <a href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/8385272?dopt=Abstract\">once<\/a>, though it was back in 1993 and the memory, alas, is fading.<\/p>\n<p>Nature&#8217;s prestige is hard-earned. The journal&#8217;s editorial staff and volunteer peer reviewers sift through thousands of submissions and select fewer than 10% for the privilege of occupying a few pages in the weekly publication. This rigorous filtration, argues the publisher, ensures that the journal serves up a quality product to the scientific community.<\/p>\n<p>The success of the journal \u2014 and its authors \u2014 is one reason cited for their resistance to open access modes of publishing. Although NPG offers several open access options through the group&#8217;s various titles and permits deposition of author-formatted manuscripts in PubMed Central 6 months after publication in Nature, it does not offer Gold open access options for the vast majority of its Nature branded journals (the notable exceptions being <a href=\"http:\/\/www.nature.com\/ncomms\/index.html\">Nature Communications<\/a> and <a href=\"http:\/\/www.nature.com\/srep\/index.html\">Scientific Reports<\/a>*).<\/p>\n<p>Alison Mitchell defends this stance by arguing that Nature&#8217;s reader\/author ratio is so high \u2014 considerably higher than \u2018normal\u2019 scientific journals \u2014 that it does not make sense to charge the journal running costs to authors, which they estimate would work out at up to \u00a330,000 per paper if a switch to full open access were to be made.<\/p>\n<p>This is a reasoned position, but I would still like to pick it apart because some of it doesn&#8217;t make sense to me.<\/p>\n<p>For a start, part of that \u00a330k fee would presumably be needed to cover the costs of the very good front matter that appears in Nature and occupies about a third of the journal. The front matter includes news, commentary, feature articles (analysing scientific trends and matters arising in science policy and education) and summaries or highlights of the papers appearing in that week&#8217;s issue). These items are mostly written by Nature staffers or commissioned from academics. There is no case for charging the costs of writing them to the authors of Nature&#8217;s scientific papers, although I would be loathe to see the front matter disappear from the journal. I suspect these are the pages that most people read. Let\u2019s face it, although Nature is a general science journal, few these days have the learning to be able to profit from all its articles. The spread is too great and the divisions between specialisms, unfortunately, are too deep (a point I will return to later).<\/p>\n<p>Part of Nature&#8217;s predicted high open access charges also reflects the very high rejection ratio \u2014 above 90% \u2014 which means that the journal processes many more articles than are eventually published. Nature relies on skilled editorial staff \u2014 at PhD level or above \u2014 and the selectivity imposed by them and their reviewers to ensure quality and maintain the prestige of the Nature brand. The careful sifting is reflected in its impact factor which, at 36.101, is one of the highest in the business.<\/p>\n<p>This latter point bears closer inspection, particularly if one has the bigger picture of science in mind.<\/p>\n<p>First, most, if not all of the papers rejected by Nature will be eventually published elsewhere, though only after the delay caused by cycles of rejection and resubmission as authors chasing impact factors work their way down the journal rankings. The chase retards the dissemination of scientific information \u2014 and can be exhausting and demoralising for authors.<\/p>\n<p>Second, despite all the careful sifting, Nature&#8217;s system is incapable of picking winners reliably, a problem that was highlighted in the past week in BBC4&#8217;s <em>Beautiful Minds<\/em> documentary on Andre Geim, who shared the 2010 Physics Nobel prize with Konstantin Novoselov for the discovery of grapheme (catch it if you can \u2014 wonderful). As revealed in the program, Nature rejected Geim&#8217;s and Novoselov\u2019s initial <a href=\"http:\/\/www.sciencemag.org\/content\/306\/5696\/666.abstract\">paper<\/a>** on graphene \u2014 twice.<\/p>\n<p>Nature&#8217;s failure in this case is not the particular fault of anyone at the journal; it simply represents the intrinsic difficulty of forecasting from a slew of submissions which ones will go on to spark the greatest interest in the scientific community. The problem is not simply anecdotal; it is widespread. Nature\u2019s impact factor is dominated by a minority of the papers that it publishes, as the journal itself has <a href=\"http:\/\/www.nature.com\/nature\/journal\/v435\/n7045\/full\/4351003b.html\">acknowledged<\/a>. A 2005 editorial revealed that fully 89% of the citations to work published in the journal in 2004 derived from just 25% of papers; at the other end of the citations distribution, over half the Nature papers from that year had fewer than 20 citations.<\/p>\n<p>It is genuinely difficult to pick winners: the skewed distribution is in fact typical of most journals, whatever their ranking, as shown by Per Seglen in a <a href=\"http:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/(SICI)1097-4571(199210)43:9%3C628::AID-ASI5%3E3.0.CO;2-0\/abstract\">fascinating analysis<\/a> performed back in 1992. Despite its rigorous selection procedures, Nature appears statistically no better at determining the relative quality of its submissions than other journals; it wins at the impact factor game because the brand ensures that the average quality of the submissions is higher.<\/p>\n<p>What this means \u2014 and this has long been recognised \u2014 is that the journal impact factor is not a reliable indicator to the quality or influence of a particular paper. Nature knows this, and <a href=\"http:\/\/www.nature.com\/nature\/journal\/v435\/n7045\/full\/4351003b.html\">publicly bemoans<\/a> the mis-use of journal impact factors in the assessment of individuals. And yet it cannot help itself from trumpeting its success in a full page advert whenever the latest impact factor calculation is published, or from dangling the statistic in the faces of <a href=\"http:\/\/www.nature.com\/authors\/author_resources\/why_publish_with_npg.html\">prospective authors<\/a>.<\/p>\n<p>But the journal is not particularly to blame for this. Their stance is all of a piece with a scientific culture that has grown to over-value journal rankings. Everyone in the business knows what publication in Nature means. It is an accolade that we seek out because the system \u2014 largely devised and run by scientists \u2014 recognises and rewards winners of this prize with funding and promotion. We might wish that it were otherwise but wishing only works in fairy tales.<\/p>\n<p>Playing the game makes fools of us all. We chase prizes that our critical faculties and our mathematical analyses have long demonstrated to be awarded prematurely and inaccurately. Worse still, running after these prizes slows us down.<\/p>\n<p>Surely we can do better?<\/p>\n<p><strong>A proposal<\/strong><\/p>\n<p>We certainly still need to weigh and judge the scientific output of our peers. This is necessary to determine distribution of funds and preferment. But rather than relying on the inaccurate shorthand of impact factors, we need to reserve judgement until <em>after<\/em> publication. I am not suggesting that we abandon pre-publication peer review, which I think serves a useful function in filtering and improving the published literature. But it would be better to fast-track publication and then to arrive at a more considered judgement of their quality by assessing how well they had been put to use \u2014 downloaded, cited, commented on, criticised or lauded \u2014 by the scientific community. Adoption of full open access would facilitate this by allowing the whole community to be involved. There is a burgeoning industry of post-publication assessment that can better harness the wisdom of the community and which, done well, could provide more accurate judgements than a handful of peer reviewers.<\/p>\n<p>We could even buttress this system by a more formal and more extensive procedure of prize-giving \u2014 prizes are important to us \u2014 which would do a better job of recognising achievement than publication in a high-ranking journal. Such prizes could replace the incentivising function of the glamour publications in stimulating the productivity of research groups. Again, if they were judged by the wider community, by the people who actually make use of the published literature, such awards would have the merit of being a fairer and more thorough assessment of scientific work.<\/p>\n<p>This is a radical proposal but it has many advantages. It is more equitable; it clears the way for open access; it speeds up the process of peer-review and publication; it accelerates science.<\/p>\n<p>The proposal may threaten the business model of Nature and break with the journal\u2019s long and venerable tradition of publishing ground-breaking work. But I think there is still a place for truly multi-disciplinary journals in this new landscape. As I mentioned already, no-one can properly read and profit from the breadth of papers that Nature currently publishes; the constraints of the abbreviated format of Nature papers and the deep specialisation that characterises modern science make this impossible. That is unfortunate for science since new discoveries are often made at the intersections between fields: Geim exemplifies this with his field-hopping success.<\/p>\n<p>I would like suggest therefore that Nature re-models itself as a platform for scientists to write a <em>separate<\/em> version of their ground-breaking research (published in long form elsewhere) that is intelligible to scientists <em>outside<\/em> their field. In doing so they would need to include the broad context of the work and to unshackle the text from excessive jargon. I can see the grimaces already but this would be a good exercise for authors \u2014 obliged to think large \u2014 and provide a valuable stimulus for interdisciplinary research to the scientific community. One might take this even further and include at least a lay summary that could be appreciated by a non-scientist readership. Nature might then rediscover its original mission.<\/p>\n<p>I don\u2019t for a moment suppose that this proposal will be taken seriously at NPG. But I thought it was worth thinking about.<\/p>\n<hr \/>\n<p style=\"font-size: 13px;\">*Thanks to Graham Steel for pointing out my omission of these titles in the original post.<\/p>\n<p style=\"font-size: 13px;\">**Geim&#8217;s and Novoselov&#8217;s paper was eventually accepted by Science. It currently has over 8000 citations according to <a href=\"http:\/\/scholar.google.co.uk\/scholar?hl=en&amp;lr=&amp;cites=9372586273470274762&amp;um=1&amp;ie=UTF-8&amp;ei=5gyUT4LiH8eZ0QXDu-zXAQ&amp;sa=X&amp;oi=science_links&amp;ct=sl-citedby&amp;resnum=1&amp;ved=0CDYQzgIwAA\">Google scholar<\/a>.<\/p>\n<p style=\"font-size: 13px;\">My thanks to the various people who sent me a copy of Seglen&#8217;s paper which I could not access from my institution.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Scientists&#8217; quest for publication in journals with high impact factors is widely perceived as one of the more refractory barriers to the fuller adoption of open access, which I believe to be in the best interests of science. But the &hellip; <a href=\"https:\/\/occamstypewriter.org\/scurry\/2012\/04\/22\/eyes-on-the-prize-are-blind-to-reality\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[152,107],"tags":[180,179,145,181,182],"class_list":["post-1497","post","type-post","status-publish","format-standard","hentry","category-open-access-2","category-scientific-life","tag-impact-factors","tag-nature","tag-open-access","tag-prizes","tag-vanity"],"_links":{"self":[{"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/posts\/1497","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/comments?post=1497"}],"version-history":[{"count":0,"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/posts\/1497\/revisions"}],"wp:attachment":[{"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/media?parent=1497"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/categories?post=1497"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/occamstypewriter.org\/scurry\/wp-json\/wp\/v2\/tags?post=1497"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}