Is Massively Collaborative Scientific Publishing Possible?

The job of a newspaper columnist is to agitate and George Monbiot did exactly that last week with a furious rant in The Guardian about academic publishers. It may have been an odd choice for most of his readers but Monbiot seemed to be actually shaking with rage as he laid into the companies that gather and disseminate the academic literature, flogging them repeatedly for charging scientists to publish, for demanding that they provide peer review services for free and, damn them all, for stashing journals behind paywalls that impoverish universities and prevent the public from accessing the fruits of research that their taxes have probably funded.

Personally, I find it difficult to think straight when I’m very angry. I get too caught up in the emotion of defending my position to see clearly and I suspect Monbiot is the same. That may explain some of his wayward swipes; he declared at one point that the academic publishing racket might even be an infringement of the universal declaration of human rights. Come on, George, calm down — you know it’s a bit more complicated than that.

And where did the fury lead? Perhaps for Monbiot it is sufficient to have stirred up the issue, but that’s not enough for those of us at the sharp end — the academics and the public.

The question of action needs to be addressed because behind the incandescent invective Monbiot actually has a point and drew sympathetic support from several knowledgeable commentators, including Jon Butterworth, Ben Goldacre, David Colquhoun and Martin Robbins. The problems are real: academic publishing is expensive for authors and readers. The scientific enterprise — and the scientific society — will work better if results are disseminated more widely, which necessitates reducing costs.

Defenders of the publishers had their work cut out in the wake of Monbiot’s article. Noah Gray made some interesting counter points from the industry perspective (he works for Nature Publishing Group), though not even he could defend charging over $30 for 24 hour access to a single article. Kent Anderson at Scholarly Kitchen tried to argue for the added value that publishers bring to the academic literature but I can’t say I found his case compelling. Part of the difficulty with economic arguments is that publishing companies are private businesses with many different products, not just journals, so it is difficult to dissect operating costs, negotiating positions and profit margins.

I don’t have the head for that kind of debate but as an active scientist I am interested in taking the matter forward. The problem of publishing is a long-standing one — Monbiot’s piece echoed arguments laid out by academic blogger Peter Coles in 2009 (though the complaints go even further back)— which means of course that it is difficult. I’m not sure I have much new to say on the subject but I was struck by how the backwash from Monbiot’s article caught on remarks made by Michael Nielsen at Imperial College this week, and I want to take the opportunity to think out loud.

Nielsen pointed out that academic publishing wasn’t perceived as much of a problem prior to about 1990. In fact, until that point it seemed a very effective way of disseminating scientific (and other) findings. But the incredible growth of the world wide web has completely changed the way we give and receive information. The primary transformation is that the web makes publishing appear so much quicker, easier and cheaper. Why therefore is scientific publishing still such an expensive business, for authors and readers alike?

For some the response to this question is that authors should take charge of the process — the do-it-yourself route. Dorothy Bishop has made some practical suggestions about self-publishing; it’s pretty easy these days to format your manuscript to look like the real thing. She and Jon Butterworth both also pointed to the arXiv database, an online pre-print repository that is popular with physicists, as a worthy model. It provides information for free (prior to peer review), establishes priority — an important concern for competitive scientists — but does not compromise the ability to publish in the peer-reviewed literature (though I have heard of some exceptions to this).

If low-cost publication and access is feasible, why is it not more popular among scientists? I was surprised to learn that arXiv is not even used by all branches of physics. In other disciplines such as my own field, the biosciences, there is no notion of an equivalent repository for research. For me part of the reason is the fear that releasing pre-prints of my work prior to publication in a ‘proper’ journal would simply give ammunition to my competitors. But high-energy physicists who use arXiv are mystified by this attitude since deposition in arXiv establishes who got there first. What are the origins of this cultural gulf and how can it be spanned?

There appears to be a clear demand for an easier publishing model (e.g. see Bjorn Brembs’ thoughtful slide-show). How else to explain the super-charged growth of PLoS One, an online open access bioscience journal that is free to readers and publishes peer-reviewed work as long as it is has been competently done — there is no threshold of interest or significance barring publication.

The growth of PLoS ONE may have been stellar but it still accounts for only tiny percentage of the total scientific output. It has changed the culture, but by how much? If cost, speed of publication and ease of access were really the primary concerns of working scientists, surely other publishers would have jumped into the market? This is happening to an extent — have a look at Nature Publishing Group’s latest title, Scientific Reports — but these new open access journals still charge a hefty fee to authors for the privilege of publication. Shouldn’t the free market be driving down costs?

The problem is that the market is not really free because the costs of publication and subscription are not the only factors in play: the key sticking point is that scientists remain in thrall to journal impact factors. Monbiot can rail against the publishers all he wants, but as long as we scientists are determined to fling ourselves repeatedly at top-tier journals — like moths to a burning lamp — we will never escape a trap that, bizarrely, is of our own devising. For who controls access to journals? Scientists. Who controls access to grants, fellowships and career progression, all of which are largely dependent on a record of publication in journals? Scientists. We know and celebrate this gate-keeping as peer review — it is a proud, though somewhat tarnished, emblem of our credibility. But are we really acting in our own best interests, both as a community and as servants of the public?

And so I turn finally to the example that Michael Nielsen mentioned in his talk, of the mathematician Timothy Gowers who, in this internet age, dared to ask on his blog “Is massively collaborative maths possible?“.

Apparently, it is.

Which of course begs the question in my borrowed title: can the scientific community work together online to improve the dissemination of its research?

We have made a few faltering steps — arXiv, PLoS ONE — but these have yet to embrace the whole scientific community. There is seepage but not yet a cultural shift, not yet sufficient resolve to break the cycle of dependency on impact factors. What will it take? A blog post certainly isn’t going to do it. I suspect we need concerted support from funders and universities, and bold leadership from a few high-profile champions — Nobel Laureates? — to take us towards a solution. It may also require the scientific blogosphere, where this issue is vigorously aired, to reach out more effectively to our disconnected colleagues.

It is easy to see difficulties — hence the durability of the problem. I thought that once I attained the rank of professor I would be free to publish where I wanted and damn the consequences. Of course that was naive; it would be unfair on junior co-workers trying to establish their own careers and would probably kill my chances of maintaining funding. Moreover, science is an international operation — the solution, when it comes, will have to be coordinated on the same scale.

However, ‘international’ is precisely the scale of the internet. The mathematicians have shown that we can use the web to amplify problem-solving intelligence. If we are too concussed by impact to do anything about our publishing woes, then we deserve every single flaw in the current model.

This entry was posted in Communication, Open Access, Science, Scientific Life and tagged , . Bookmark the permalink.

113 Responses to Is Massively Collaborative Scientific Publishing Possible?

  1. Tom Hartley says:

    Personally I think governments should fund a mechanism to pay a fixed (low) fee for article processing in Open Access journals for any work meeting a certain (low) quality threshold and is wholly or partly public funded. Alternatively they should set up a no-frills journal/repository which fulfils this function. It should allow text figures to be formatted based on markup code similar to wikipedia. Tools for automatic content- and social recommendations and post-publication review and commentary should be developed. We should move away from journal impact factors and directly measure the readership, citation and peer-recommendations of individual articles.

    Government has the resources to do this and the motivation, since it is largely government money which is subsidising the publisher’s huge profits at the moment.

    Any commercial journal which can survive the competition (e.g., by offering greater impact, selectivity, better peer review, better search/recommendation) would be free to charge subscriptions and limit access as market conditions dictate. The bulk of commercial journals are unlikely to survive since they fail to add sufficient value to the articles.

    Research funders should also consider charging commercial publishers for the raw material they currently provide for free.

    • Stephen says:

      The thing is, I think the change has to be wholesale and concerted. We almost have the situation that you describe as desirable but I think that’s still too fragmentary. We don’t just want to make the lower-grade reports accessible (PLoS ONE may be a business success but the quality is very variable) — we want the good stuff too! The move away from dependence on impact factors will take a seismic shift that involves not just funders but employers too.

      • Tom Hartley says:

        I agree that is the problem. I have been banging on about this for ~10 years, and find it frustrating that scientists don’t take things into their own hands and create something better than the current system. The reason I stress the role of government is that I believe that where they lead, funders and employers will follow. For example, the government could easily change research assessment criteria which currently drive people to publish in high-impact commercial journals. I don’t believe it is beyond the wit of man, and indeed many of the necessary steps have already been taken. FWIW, I think George Monbiot’s article was only very slightly over the top, and a robust critique of the academic publishers’ business models is long overdue.

  2. Steve Caplan says:

    Great post examining the complexities of scientific publishing. As you note, not all peer review is equal (or is conceived as equal). Given that impact factor is basically a formula that calculates the average number of citations of papers in a journal over time, then one would think that if a scientist publishes in a low impact journal, but in two years garners >100 citations, this would compensate and make his/her paper higher impact than the average Cell/Science/Nature paper. But would anyone in the biomedical sciences swap a Cell/Science/Nature paper for a BBA paper with a high number of citations? I think not.

    What this means is that you are absolutely correct in noting that seismic changes in the landscape would be needed to change the current system. We continue to aim for higher impact journals NOT because their average impact factor is higher, but because these journals provide us with a cloak of respectability in the scientific community.

  3. Heather says:

    My goodness, you write great blog posts. If any post *could* break the cycle of dependency, it would be one of yours.

    “I thought that once I attained the rank of professor I would be free to publish where I wanted and damn the consequences. Of course that was naive; it would be unfair on junior co-workers trying to establish their own careers and would probably kill my chances of maintaining funding. Moreover, science is an international operation — the solution, when it comes, will have to be coordinated on the same scale.”

    You also lead right up to the most important point – as you underline in your response to Tom above. I once thought exactly the same as you, above. I even thought I might self-publish. Hell, my results were going out in my online lab notebook. But those junior co-workers in my and my collaborators labs are held hostage – by our own devises. I’m not on career selection committees yet but I’ve already vowed to support alternative career paths well defended and complemented with convincing job talks, actively. Now that’s on public record, I won’t *get* invoted to career selection committees, perhaps…

    One point you don’t raise, that could be important to introduce, is that of the tribune. We all have a limited amount of time a day. The cultural shift will have to be enormous to get us to shift from our current reading list of a dozen or so journals in our field, which we read, and in which we tend to publish to reach the colleagues with whom we need to communicate, to be recognized and our work taken into account in theirs, later. It’s already much to add or subtract one to that list. Imagine substituting the lot with a new venue? How long would you need to read all of the above, to catch the work in which you are interested?

    Of course, as a generation of scientists moves away from Tables of Contents and more toward keyword searches using tools that already exist, it won’t matter any more. The serendipity factor will be there in other forms.

    Far more incoherent thoughts than yours. Thanks much, Stephen.

    • Stephen says:

      Cheers Heather – I think you have already hinted at the solution to the problem of how to wade through the literature if it is simply placed in an undifferentiated repository. We can develop search tools and, as Mendeley and others have done, could harness the power of the community to flag up articles of interest. Of course, it’s not going to be quite as simple as that…

  4. Joe D says:

    “In other disciplines such as my own field, the biosciences, there is no notion of an equivalent repository for research.”

    There is an equivalent — Nature Precedings (http://precedings.nature.com/)

    Of course, you are correct that there is no notion of an equivalent — with about 4,000 pre-prints in 4 years (now dwarfed by the annual output of plos one alone), it doesn’t seem to have caught on.

    • Eva says:

      I have a manuscript in Nature Precedings, deposited right after I finished my PhD, but only *because* that was unfinished work and nobody was going to finish it and my supervisor would never get a “proper” publication out of it. It’s a great place for the things that don’t otherwise have a home (and at that level, Nature Precedings is a more attractive option than one of the journals that explicitly say they focus on negative results…)

      • Stephen says:

        Eva – do you know if Nature Precedings papers are listed on PubMed?

        • nico says:

          Short answer – no. I understand it has to be peer-reviewed to get into PubMed, but I could be very wrong.

        • Eva says:

          What nico said. They’re not. But they’re also not negatively affecting impact factors of high-impact labs – something a negative results journal (listed on PubMed) *would* do.

          They’re citable, though, and more public than theses (my dissertation is also free online, but who’s going to read *that*?!) so it’s useful if you effectively just want to free your data when you leave the lab.

          • Eva says:

            read “h index” for “impact factor”. I was thinking faster than I typed there, but you know what I meant.

          • Stephen says:

            That’s a shame because surely it makes the research harder to find? I rely quite heavily on RRS feeds from Pubmed, for example and that will miss anything from Nature Precedings.

            • Google Scholar does index Precedings (including citations to it) so for instance you can find http://bit.ly/ravuGI where the Nature Precedings version has been cited but the book chapter has not (I think). But I admit I don’t use GS for searching until I’ve exhausted PubMed myself.

    • Stephen says:

      Thanks for reminding me of the existence of Nature Precedings (which somehow got lodged in my mind as a placed for people to publish conference posters) and for pointing out the limits of its success. It has suffered, I suppose, because of the culture within the life sciences that frets about being scooped and ensuring work is published in high impact journals.

  5. Steve Caplan says:

    Just an aside about PLoS One, as I happen to be one of the many academic editors with the journal for the past few years–the amount of papers and workload is absolutely incredible. There are weeks when every morning I find myself asked to handle 8-12 papers. If I agreed to handle even 1/10 of the number of papers I am sent, I would be doing editing full time. I don’t know the master plan, but my feeling is that this model cannot hold. Something has to give–if content becomes compromised due to inability to complete proper review, this will spell serious trouble for such a publishing model.

    • Stephen says:

      OK, that I didn’t know. And it is astonishing!

      You’re right that such a massive increase in volume is unsustainable. This raises the not unrelated issue of whether the community should also contemplate a shift to post-publication peer review, which some have advocated.

      I’m not sure where I stand on this. It hasn’t worked in the past but that’s not to say that it can’t ever work and I suspect that increased habituation to the mores of social media will propel things in that direction. However, I can see downsides: for example, I imagine that the medical research literature would get polluted by a lot of log-grade stuff on highly questionable alternative medicine practices such as homeopathy. Though against that one sees plenty of papers on these topics in the peer reviewed literature (mostly in specialist journals where the peers are other homeopaths).

      This takes us into the territory of reputation management which is an important issue — and has been raised below by Ian Hopkinson. Peer review is a more or less closed system for doing this and perhaps gives the scientific community a measure of control that they would be reluctant to give up by ceding peer review to a comment thread beneath the published paper.

    • Eva says:

      [puts on Company of Biologists hat.]

      Hopefully, with some new journals, such as Biology Open (launching in a few weeks), accepting a similar kind of papers as PLoS ONE does, the load will be distributed among multiple journals.

      [takes hat off again]

  6. Nathan Brothers says:

    The issue is complex and this article does a good job of voicing some of the complexities, but I believe this article and the piece from the Gaurdian are oversimplifying the apparent dichotomy of “open v.s. closed” publishing.

    Closed publishing is portrayed as an unnecessary drain on research funding at the expense of libraries, independent researchers and the greater public – nothing more than money to line the pockets of the rulers of an archaic social system propped up by the marble columns of the academy.

    Open publishing is described as a novel “grassroots” alternative that will not only threaten the old models, but will almost certainly put the STM publisher barons out of business once the scientific community comes to their senses.

    This couldn’t be further from the truth. From the perspective of a for-profit publisher, open access publishing have significant business advantages when compared to traditional journal publishing and very few disadvantages.
    1) With competitive pay-to-publish pricing an open access journal can be equally, if not more profitable to a publisher than a closed access journal.
    2) Due to higher average dowload rates, open access journals tend to have higher impact factors than journals with comparable publication quality.
    3) The impact factors of newly launched open access journals tend to improve more quickly than newly launched closed access journals.
    4) Lanching a journal as open access mitigates a substantial amount of the risk of starting a new journal, because the journal’s profits do not depend on Impact-dependent subscriptions.
    It might be years before a new closes journal accrues enough subscriptions to break even, much less turn a profit. Whereas an open access journal can become profitable in it’s first year simply by publishing enough articles and collecting author fees.
    5) By estimating the average lifetime subscription value of an article if it were published in a closed journal, a Publisher can select a open access pay-to-publish fee that is equal to the article’s hypothetical lifetime subscription revenue. Because having $365 dollars today is strictly better than receiving $1 each day for the next year, this is very good for business.

    I certainly believe that open access is a more efficient, egalitarian, and productive publishing model; but it’s not all roses and certainly is not going to reduce publisher profits make the publisher obsolete.

    (I’m a Product Development Manager at Springer Science+Business Media, the world’s largest open access publisher)

    • Stephen says:

      Nathan – many thanks of providing an informed publisher’s perspective. I take the point that my post did not cover every nuance but there are many intersecting issues here and it was already too long at over 1200 words.

      It is interesting to see the positives of OA publishing from the industry point of view. However, you don’t address what for me is still the key issue — how to break the stranglehold of impact factors which remains an impediment to better dissemination. I know that top-tier journals do provide OA options for authors but they can be expensive. Cell and Nature charge up to $5000 for the privilege, in contrast to PLoS ONE’s $1350 fee. Any idea why their charges (which are independent of any charges for pages or colour figures) are so much higher? Is this market forces pure and simple — and so another indication of the obsession of scientists with IFs?

      I’m not against differentiation of quality between different publications, I hasten to add. It’s just that I would like to explore cheaper ways of achieving it that can also enable free public access.

      • nico says:

        You will have to note that the PLoS fee is heavily subsidised, to the tune of “$9M start-up grant from the Gordon and Betty Moore Foundation and has received support from the Sandler Family Supporting Foundation, the Open Society Institute, the Irving A Hansen Memorial Foundation, the Doris Duke Charitable Foundation, the Ellison Medical Foundation, the Burroughs Wellcome Fund, the John D. and Catherine T. MacArthur Foundation, and many other foundations, universities, and other organizations and individuals” so $1350 isn’t the “market price”.

        At Nature the price is higher partly because all our production staff is in the UK/USA/Japan (many journals have outsourced their copy-editing to India), and we also have an art team that redoes all the figures to make sure they look their best. Plus we have a News team, whose output you can read for free (with registration now)!

        The price I have heard mentioned to produce a medium impact paper hovers between 2500 and 3500 GBP depending on subject (we talk science here, I guess English lit journals do not have many figures etc!), to put the Nature price in context. BioMed Central is cheaper, but I understand that very little copy-editing happens there; great for people who can write in the good Englich, not so great for the others.

        Also PLoS and BMC publish online-only, whereas Cell and Nature send you an amazingly high-quality paper magazine for you take to the loo!

        • Stephen says:

          OK but that raises some, perhaps cheeky questions. PLoS may be subsidised but since it is usually funders who pay page and OA charges, isn’t that just another form of subsidy?

          And why should authors of original content for Nature be expected to pay for your News team (however talented)? ;-)

          The work you guys do on figures may be costly but have you considered offering discounts to authors who submit camera-ready copy that conforms to your exacting standards? I’m sure that would be popular (and that some could manage it).

          And finally (and completely off-topic) — how is it Nico that we have never met? I now realise that it’s because neither you nor Henry has had the good manners ever to invite me to lunch at Nature Towers! ;-)

          • I’ve argued in the past that the argument about subsidy is a spurious one. All academic publishers are fundamentally in receipt of massive direct and indirect subsidies (payment of subscription or author fees and costs of reviewing). The question for the subsidiser is how to most efficiently spend that subsidy/investment in the communication process. Framed that way the discussion actually becomes quite different. And it winds up a certain type of publisher which I will admit I find quite fun :-)

            I’ve always felt that NPG got their cost structure the wrong way around. They should charge for the print edition and in particular the news articles and make the science free online rather than the other way around as they have done (free news online and charge for the science). In terms of the argument above should governments and research funders be subsidising what is essentially a popular magazine vs the effective communication of the science? There are arguments on both sides of course but it seems to me the question bears asking.

          • nico says:

            Nature being the highly sought-after venue that it is, it could probably get rid of its News team, and still be highly attractive (a la Cell). But can you really imagine Nature or Science without the News bit? It is part of the overall strategy and brand that we do have a good (in fact, multiple award-winning) news team. Isn’t it great that we have an Adam Rutherford doing his thing on the BBC/YouTube Nature channel, or Geoff Brumfield on the podcast (for example)?

            It would be interesting to trial that discount idea, but images really are quite complicated and although I have no doubt there are many scientists capable of producing exactly what we need, from experience there are many more who aren’t. This isn’t a denigrating comment, I am completely incapable of doing what the art team does too! All in all, it is just easier to have a single price structure, it makes quality control and last minute changes easier too.

            I believe we have never met because I am one of those bots we are introducing to improve the editorial process, I live in a server in the bowels of Nature Towers ;-)
            No really if you want to pop in, just drop us a word, we’ll give you a tour and coffee!

          • Frank says:

            Minor point – for most Nature-branded journals there is no fee to publish. They allow you to self-archive (and will deposit into PubMedCentral for you) but do not have a fee-paid, immediate open access option. Nature Communications is the exception to the rule, and of course the new Scientific Reports is a fully OA journal.

            Turning to Nature News, and the similar offering from Science magazine, it’s worth remembering that the issue of online news is another contentious area of discussion. To pay or not to pay. Does erecting a pay wall to online news take you out of the online conversation? Nature have tried different approaches. A few years ago Nature News became a premium service that institutions had to subscribe to if they wanted access. It then became free again, I am not sure for the reasons.

      • Tom Hartley (@tom_hartley) says:

        At the risk of restating the obvious, the costs of processing an article for a journal are vanishingly small compared to the costs of doing the science, writing and reviewing the paper, yet the journals get all this from academia (and charities) for nothing while making a profit from charging academic authors (OA) or from academic subscriptions. This is so bafflingly different to any other economic transaction I can think of that it is hard to see how it could ever be sustainable. Why wouldn’t academics/funders/government set up non-profit journals that could fulfil the same function and cut out the middle man? Why don’t we? As Steven says the current situation is only sustained through momentum – no-one would ever opt-in to this model if it didn’t already control the market.

        • nico says:

          Ther are indeed non-profit publishers out there. My previous employer, SGM, is a charity, and as such they use any surplus from their publishing activities to sponsor conferences, outreach to schools, book grants for developing world institutions, etc. If they were to cut their prices, they wouldn’t be able to carry on with their charitable activities. All their journals are available free for anyone with an IP address in a country classed as “developing” by the UN, reduced prices for intermediate, and full price for OECD countries, which seems quite a fair compromise to me. They also have an OA option, and papers in JGV, Microbiology and JMM are free after 12 months, papers in IJSEM after 24 months.

          They are also unusual in that they still run all their operations in house from Reading, with just around 30 staff (including editorial for the 4 research journals+Microbiology Today, membership, conferences…), they are busy I can tell you!

          Most societies nowadays have subcontracted the publishing activities to for-profit publishers, and they seem happy this way. There is nothing stopping the learned societies around the world from taking over their publishing activities again, or for institutions and universities to do the same. Yet, they don’t (excepted the recent Wellcome/HHMI collaboration), so I guess commercial publishers must be giving them a pretty good deal.

      • cromercrox says:

        PLoS is subidized by charitable donations (I think). Nature and Cell aren’t.

        • Stephen says:

          Except perhaps when charities like the Wellcome Trust commit to paying OA charges on behalf of authors…?

          Wellcome are actually very good at this and set and example that I wish Research Councils would follow. I think I’m correct in saying that RCs don’t pay for OA if you submit your paper after the end of the grant, which is often the case. Would be happy to be corrected on this if I am wrong.

          • Frank Norman says:

            Stephen – yes that is correct. MRC instructs grant applicants to include OA costs in their application. Usual rules on spending grants apply. I think RCs are restricted by Treasury rules, so cannot be as flexible as Wellcome.

            I presume that University accounts departments could do something creative to extract OA funding from the grants.

  7. Pingback: Ninth Level Ireland » Blog Archive » Is Massively Collaborative Scientific Publishing Possible?

  8. I think it’s a mistake to see scientific publication now as a dissemination problem.

    The problem is now a “reputation” problem, we determine our reputations in part by the institutions that employ us and in part by the journals which publish our work. We have helped to build the reputation that journals can give to a paper by providing editorial work and review for free.

    If journals charge for access we should charge for the work of peer-review.

    There is no technical reason why we can’t have an online reputation system, the real block to this is the level of participation – traditional journals have a 350 year start on us, everybody takes part in the game. If you look across your departments: for how many would an online reputation system work?

    I wrote about this a while back:
    Publication, publication, publication

    • Stephen says:

      Thanks for your comment Ian. I agree – and disagree. I think both problems, dissemination and reputation, exist and that they are linked. But if we can find a way of rewarding good science (i.e. good scientific publications) that is independent of getting published in high impact factor journals, both problems can be solved together.

      The question is, how do we get there. The longevity of journals is a factor in their favour but these things can change. The 350 year figure you mention I presume refers to Proceedings of the Royal Society (or ?) but this is no longer regarded as a top drawer destination — it has been supplanted by newcomers such as Nature (not even 150 yrs old!). Perhaps online reputation is the way to go — I was alerted this morning by @sharmaedit to this interesting note (pdf) on the rise of impact metrics based on social media — but the way forward is not yet clear to me. Most scientists are simply not online (in that sense) as yet.

  9. nico says:

    Joe beat me to point out that there is a biological alternative to arXiv, Nature Precedings. The tool is here and it works well. Note that “Nature journals will consider manuscripts that have already been circulated as preprints, but some other publishers will not, and many journals have policies that are vague or ambiguous on this point”. So publishing in NP does not preclude publishing in other NPG journals.

    Tom said “The bulk of commercial journals are unlikely to survive since they fail to add sufficient value to the articles”: I don’t know about that, I have worked only for a handful of journals (JGV, IJSEM, Microbiology and Nature) and I can assure you that at each and every one of them the editors/staff work hard at making every paper as good as possible.

    • Stephen says:

      Cheers Nico – I have no doubt most people working in publishing are working hard to improve the quality of the scientific literature — even if I do get irritated from time to time by editors setting deadlines for me for work I am expected to do voluntarily! ;-).

      But you raise an notable difficulty — one of the factors inhibiting authors from using repositories like Nature Precedings is confusion about the rules on how that might affect future publication in a peer reviewed journal. Nature’s position may be clear but there needs to be a universal standard since no-one can guarantee publication in one of your prestigious titles and we always need to have a plan B (and sometimes plans C, D, E and F…).

      • nico says:

        I sympathise with you re. deadlines, but we have observed that if no deadline is given the time to review increases slowly towards infinity…

        I understand the reluctance to submit to Nature Precedings too, but someone has to start sometime, and we can impose editorial decisions only on our journals. I believe most journals would be happy to accept papers that have been through NP or others first, and if not well, do you really want to publish there?

  10. Hi Stephen,

    Good post. After writing mine which you link I above, I was a bit surprised to find out that freely distributed preprints are allowed by Nature and others. In my opinion this puts the ball in the court of scientists to use these servers, as you are saying . This is a somewhat different discussion from the value (or not) added by journals to peer review.

    I’m not sure it takes a seismic shift to do this, but it does mean individual researchers have to habitually do it. This would mean that e.g. news articles could link to the preprints routinely. If they do this, its likely the preprint server would become the primary source, as arXiv is in my field. With arXiv, this happened rather gradually over several years, pioneered in fact by the theorists. We experimenters were slow adopters.

    It is also true that “open access” deals can be very expensive to the submitter, I’m not sure they are a panacea.

    Jon

    • Stephen says:

      The ball is not yet entirely in the court of scientists since not all journals, if I understand correctly, will consider manuscripts that have been published as a pre-print (see Nico’s comment above).

      However, I would agree that it is in our court to the extent that it’s up to us to agitate for that change of culture. Maybe seismic shift was too grand a metaphor but a change of habits will not come easily — as the past two decades have shown. As a physicist, do you know why some physics disciplines make less use of arXiv? If we could unpick that, there might be more hope of getting the chemists and biologists to consider it more seriously.

      You’re right also that there are several issues tied together here under the banner of figuring out how to disseminate research more freely — as this comment thread is already demonstrating!

  11. nico says:

    There is a great post at the Scholarly Kitchen: http://scholarlykitchen.sspnet.org/2011/09/08/rallying-cries-vs-reality-profits-and-publishing-meet-academics-and-idealism/

    I think post-publication peer-review is not going to work unless there is a clear system for reviewers to earn recognition through it. Right now it would score just as high as a blog in the eyes of funders/employers, i.e. approximately nil.

    About the cost to access papers, there are many ways from emailing the author to inter-library loans etc. It is expensive, but then digitising and maintaining online repositories isn’t cheap (the Nature back catalogue goes back to 1859, I have no idea how much that cost but I guess “a lot”). And publishers (whether non- or for-profit) do try to innovate, for example by including links to genes and proteins in curated databases, help in finding related papers, etc. At some point someone has to pay, and I very much doubt the system we have now is the least effective. It could probably be better, but then so can everything.

    • nico says:

      tyop correction: 1869, not 1859

    • Stephen says:

      Nico – that’s another point worth emphasising. I hope I haven’t given the impression that I think the internet makes everything ‘free’. Clearly there are real costs. I mainly want to explore ways of keeping these to a minimum.

      With regard to the point about post-publication peer review, is recognition for reviewers the key? After all, they are mostly currently anonymous and don’t get any recognition for their efforts. Unless you mean that the recognition is currently in terms of the unspoken quid pro quo in which reviewers take on assignments on the understanding that others in turn will review their manuscripts.

      If the review system were to switch to more open, post-publication forms, that would certainly entail a big shift in attitudes. The initial experiments have not been a great success I understand, in part I think because of a fear of repercussions if one is too critical. The solution to that is anonymous comments, but we all know what sorts of abuse that is open to.

  12. cromercrox says:

    I think George Monbiot must have had a paper rejected. In any case, Nature is massively ahead of the game – just check out today’s announcement – http://www.nature.com/nature/journal/v477/n7363/full/477244a.html

    But seriously, I am angry, too. I am angry that the people who shout most loudly in this debate simply don’t understand the role of professional editors (disclaimer – I am an editor with Nature and have been so for almost 24 years).

    We read all the substandard, not-quite-good-enough, sort-of-okay-but-not-Nature manuscripts, so you don’t have to. This is a massive task (Steve Caplan alludes to this above.) If we weren’t there, you’d have to wade through it all yourselves, with no initial filtering – as Heather says, you don’t have the time to do that.

    Yes, it’s imperfect, and fallible, but I’d say that it works well >99% of the time. Trade that off against the effort of having to wade through oceans of dross to find a paper that’s good, or which doesn’t present misleading results, consideration of which could waste years of your research time chasing will-o’-the-wisps.

    We editors in top-tier journals deserve our reputations because we work hard to ensure that the papers we publish are the best. If we have such reputations, it is because we have earned them.

    • Stephen says:

      Henry, what you say is certainly true of Nature, but further down the food chain, the people doing most of the reading are the reviewers — though, to be honest, I’m not entirely sure exactly how much triaging goes on at other journals before papers are sent out for review.

      • cromercrox says:

        I can’t speak of what happens at other journals. At Nature, though, we editors put a hell of a lot of work in, not just at the initial ‘triaging’ stage, but – the the advice of referees – helping the authors develop their manuscripts – this can take several iterations. As far as I know, authors (whose papers have been accepted, at any rate) are grateful for this work. Revising a paper is agonizing at the time, but the results are almost always worth it. And it does a journal’s reputation no good if it routinely published half-baked papers anyway… A related issue is that pressure to publish (in any journal) tempts authors to publish small slices of preliminary work rather than taking the time to do a more rounded study that’ll be of more use to the community. Editors work against that trend. Without editors, there would not only be a vast amount of unfiltered material o variable quality on the web – but it would be in very small slices, so readers would have to do more work to get the full picture.

        • Stephen says:

          I have no doubt that everything you have said above is true. Of course, editors aren’t the only ones ensuring quality and trying to prevent excessive salami slicing of results — peer reviewers do it too. I accept that editors at top-notch journals such as Nature are probably more involved in effective quality control that elsewhere.

          And whatever the mechanism, it is certainly true that the triage/review system does improve the published product. I have experienced that benefit myself (and hopefully also given it to others). This aspect of the process is primarily what makes me hesitate over post-publication review.

  13. cromercrox says:

    What’s more, post-publication peer review doesn’t work. The general absence of comments on papers (including PLoS papers) is testament to that. The same people who think peer-reviewers should be paid are presumably the same people who would selflessly spend hours and hours reading posted submissions (because reviewing a paper is not a trivial matter), in their own time, and without being paid, in order to air their pearls of wisdom on the web – and there would (presumably) be no checks on whether such commenters had no other motivation, such as a conflict of interest or an axe to grind with the researchers (hey, it happens.)

    In my experience, which I am bound to say dwarfs that of many commentators on this issue (arrogant, I know, but true), referees of papers work unbelievably hard, and they do it for nothing – because they expect that their own submissions will be treated in the same, respectful way.

    • Stephen says:

      Fair comment henry, but just because it has failed in the past, doesn’t mean it won’t ever work. That said, I still don’t know where I stand on post-publication peer review. I completely take the point that it would mean a lot more dross ends up in the literature (though there is plenty of questionable dross already out there even after peer-review. Arsenic life, anyone…?).

      I think people can judge quite quickly from an abstract whether a paper is of interest to them. If it’s relevant, they’ll read on. And if they find that it’s good, or a serious flay, why wouldn’t they feel motivated to post a comment. Especially if the zeitgeist came (as with blogs) to be that there is a courtesy of commenting on each other’s stuff. People still discuss papers at conferences — this would just take it online. Of course that does not deal with the problem of whether commenters should be anonymous or not…

      Nature started out as a community effort, did it not? As a way for the scientific community to disseminate its results more systematically. I wonder are their ways for that process to be reborn with via the opportunities offered by the world wide web.

      • Steve Caplan says:

        Post-publication peer review could NEVER work in the biological sciences. Today the publications are so highly specialized, that in order for me to build on findings even slightly outside of my realm of expertise, I need to depend on the publication being scientifically sound and properly interpreted. In many cases outside my own realm of expertise, there’s no way I would be able to do that on my own without actually inventing the wheel.

        • Stephen says:

          But if you are reading a paper that is linked to your field of research, don’t you make your own mind up anyway about it?

          Today — with peer review — when I read results based on techniques that I don’t understand, I’ll usually consult a colleague who does for a view. Couldn’t such queries be made online as well — in the brave new world that I’m struggling to imagine.

          But maybe i’ve missed your point…

          • Steve Caplan says:

            The problem is time. Many things that I read seem logical, and I can understand the experiments, but do I know that the controls used are the most prefered by those in the field? I don’t have the time to chase down experts in every field (and my institute is not huge, so everything would have to be done by e-mail).

            When I read a paper, I really want to be able to rely on the technical aspects of the work. Sometimes the interpretations may be up for grabs, especially if I know of unpublished data from my lab or others that might factor into the overall interpretations. But I would hate to have to wade through a pile of crap and rely on a string of threads to determine if conclusions from experiments are accurate or not. It would slow my kind of work down to a knee-less crawl.

      • cromercrox says:

        Stephen – as much as one would like to invoke Schadenfreude about the Arsenic Life paper, these cases make the news in part because they are so rare. I suspect that the amount of ‘dross’ in top-tier journals – and, I submit, journals generally – is rather low.

        • Stephen says:

          True for top-tier journals but plenty of dross has been published elsewhere after peer review. I had the misfortune of reviewing a paper a few weeks back that was so bad I couldn’t even give it the nod for PLoS ONE. But I fear it will eventually emerge somewhere…

  14. nico says:

    To the kind people who review papers for Nature, know that the three most common requests I get to break our style rules are:

    1. Add more “equal contribution” authors
    2. Add more corresponding authors
    3. Acknowledge the reviewers and the editor (hear that Henry?) for improving the paper

    Regretfully I have to always turn down no. 3 (journal policy), for nos 1 and 2 we can talk…

    • Stephen says:

      Seems rather unfair on your hard-working staff!

      • nico says:

        I guess one reason we do not allow it is because that’s what we’re paid to do, so a self-congratulatory pat on the back seems a bit misplaced. I sympathise with the reviewers, their work is unaknowledged but essential.

        I know of someone working in the industry that collects emails of praise from authors though cough *not me* cough.

  15. Matt Wall says:

    My own small act of rebellion against the publishing-paywalls is to make sure that all my papers are available as PDF’s on my own personal website – they seem to get ‘found’ and linked to by Google Scholar, which is great. I’d encourage everyone to do the same, if possible.

    • cromercrox says:

      Your act of rebellion is indeed small. At NPG the author retains copyright anyway. Up the revolution, eh?

    • Stephen says:

      You have to be careful with that since it may be in breach of agreements that you have signed with the publisher…

  16. apgaylard says:

    This is a very thoughtful contribution to a tricky debate. The perils of inconsistent rules on what constitutes prior publication from a journal perspective would need to addressed if repository or personal publication became more widespread, to allay fears that making material available might prejudice getting a decent journal paper later. My own trivial brush with this was having a letter to a 5th rate homeopathy journal rejected on the basis of prior publication of the comments on my 100 hit/day blog.

    As to whether journals add value and by how much, I can appreciate that this is considerable at the top-end, as the commenter from Nature argued. I do have a problem seeing much added value at the lowly journals where I publish.
    My work is mostly engineering and the most read journal in my discipline is produced by the Society of Automotive Engineers. This uses volunteer labour to select from contributions offered, perform the peer review and manage the process of getting a ‘print ready’ manuscript from the author. Templates are provided for drafting and ‘type setting’ is done more or less automatically. This points to fairly low added value.

    On the other hand the SAE have developed the software and publications platform; they also provide on-line peer review and submission management tools. Finally they made the finished product available on-line and in print. There are certainly costs to be defrayed and investment that requires a return.

    Different publishing (dissemination and reputation) models may well have other effects. As Kuhn pointed out, journals have a historic role in defining scientific communities. They draw together readers and writers with often very specific shared interests. I wonder if repository or self-publishing would do the same? Though it may be that the same ends could be accomplished by other means. If repository or self publishing became more influential I wonder if that would create opportunities for ‘virtual’ journals that indexed or collated work on specific topics. Perhaps models for these already exist?

    I guess I remain somewhat conflcted on the issue.

    • Stephen says:

      Thanks – many good points. I certainly acknowledge that there are real costs that should be borne to ensure that the scientific literature is disseminated well and properly archived. For that reason I would much prefer international repositories to self-publishing (or even institutional publishing).

      Kuhn’s point about the journals helping to define communities is a very interesting one. I think the collaboration to set up online variants would achieve much the same end — just with different technology.

  17. Peter says:

    I’m not a scientist (just a lowly postdoc in mathematics), but I keep wondering if a real solution could (should?) lie outside of scientific publishing. It seems to me that the major problem is the monopoly publications have when it comes to assessing researchers. We’re all (mathematicians included) obsessing over metrics to assess publications which tell us very little about a researcher’s qualities, especially a young researcher without much of a record.

    Gowers’s Polymath project is/was wonderful, but rather traditional in that sense. I think the biggest change in research mathematics in the last two years has been the rise of mathoverflow (which led to stackexchange creating more research related sites as well as similar platforms developing elsewhere). On mathoverflow, researchers (including Tim Gowers) ask and answer each others (research level) questions. Mathoverflow is not without flaws, but it offers an additional way to contribute to the research community — at the highest level.

    I believe that within a few years, the activity on mathoverflow will become a serious factor when it comes to hiring decisions simply because it completes the picture of researchers regarding both the research quality and how a researcher interacts with the community.

    The point I’m trying to make is: publishing is one way to contribute to the research community. Can there really be no others in the sciences?

    —-
    PS: Incidentally, Gowers also started another project earlier called tricki which is dormant, but fascinating. I remember a discussion on Occam’s Typewriter about protocols and the right kind of tubes that could fit right into such a project.

    • Stephen says:

      Interesting stuff. Had never heard of Mathoverflow — but then I’m not a mathematician. You guys seem to be ahead of the curve! Of course much practical science can’t be tackled online in that way, but here I’m primarily concerned with the process of review and publication which can. I hope we can do better…

      • Peter says:

        Thanks for adding a working link. Mathoverflow is explicitly not for open questions (i.e., you shouldn’t ask for solutions to your personal research); its community views it more like a big departmental common room (have you read that paper, how do you understand xyz approach to…) .

        I think the net can help transparently measure contributions to the research community beyond papers and I think this will be the key. Dead-tree editions are not the best tool to document research activities and results anymore. I think I would trust a specialized, transparent social networking tool more than than any publication metric to assess a researchers contributions to a field.

        It may seem impractical to develop such tools right now (“Who needs that right now? I have papers to write!”), but so did the arxiv when it started.

  18. cromercrox says:

    A fifth-rate homeopathy journal? By the rules of homeopathy, that’s a pretty powerful dilution. Does one have to bang the journal sharply on the edge of one’s desk before reading?

    Sorry. Couldn’t resist.

    • Does one have to bang the journal sharply on the edge of one’s desk before reading?

      I think that when reading a(ny) homeopathy journal it is one’s head that mostly ends up being banged on the desk…

  19. Mark Pallen says:

    Great blog post and discussions. You might be interested to look at a recent adventure in Open Access Genomics, where we managed to crowdsource some analyses of the German E. coli outbreak strain, collated on a wiki:
    https://github.com/ehec-outbreak-crowdsourced/BGI-data-analysis/wiki
    and still managed to square the circle of getting a publication on this effort in a high-impact journal, despite the supposed existence of the Ingelfinger rule.
    http://www.nejm.org/doi/full/10.1056/NEJMoa1107643
    http://en.wikipedia.org/wiki/Ingelfinger_rule
    Sadly, although open access before appearance in the deadtree version of the journal, our paper is now available only to subscribers.

    But nonetheless, I would like to think it sets a promising precedent.

    • Stephen says:

      Thanks — that’s a really nice example. The more we see of this sort of thing in the practice of research, the more the same philosophy will seep into the processes of publication.

  20. Stephen says:

    For completeness – the inimitable Cameron Neylon has commented on this post over at Google+.

  21. cromercrox says:

    Stephen – somewhere in the above you said that Nature started as a kind of community journal. So it did – it was, kind of, the house journal of Thomas Henry Huxley and his London cronies. But this didn’t come for free. It was capitalized by the seemingly bottomless pockets of the publisher Alexander MacMillan, and didn’t turn a profit for ten years. I think this shows that no matter how starry-eyed you might be about open access, there will always be a trade-off between quality and price.

    • Stephen says:

      Nonsense, I’m sure there’s a solution. We just have to look harder. Hang on — I’ll get my telescope…

      • cromercrox says:

        Apparently there’s a jolly nice supernova in Ursa Major you can point it at. And Jupiter is bright in the East at the moment.

  22. Tom Hartley says:

    On twitter Stephen points out that the comments have not dealt with how scientists and academics can break their dependence on journal impact factors.

    The problem is the way IF is used (implicitly) as a proxy for quality in REF/RAE etc., written and unwritten promotion criteria and general kudos.

    The solution is not to rely on subjective peer (old-boy) ratings, in my view.

    We should move to article level metrics: readership, recommendations and citations.
    We can also look at individuals’ influence in the same terms.

    People protest that citations and readership figures do not accrue quickly enough, but recommendations are a solution to this. For a model look at f1000 – which is excellent in my view. To make this work properly we need to establish an even bigger academy/faculty (e.g., you might qualify by having a PhD in a particular discipline, or by having X peer-reviewed publications). This will avoid the biases inherent in judgements made by individuals and small groups. It’s a kind of post publication review. If done properly it would also be the first place to look for new publications in one’s field.

  23. I think an interesting question here is, why isn’t there a high impact open access journal? My perhaps naive viewpoint is that if one of the open access journal providers wanted high impact they just need to get more choosy about the publications. Nature/Science have high impact factors as they publish very good work, but also very few papers. Most of the current open access journals that I am aware of publish papers in their thousands and are not that discriminating about the content. I assume that this is purely a financial decision. Would a much more selective OA journal be able to survive if it published only 25 very high quality papers each month? And then, of course, you have to get people to submit to such a journal when it has no brand.

    Very recently we got a paper bumped from a Nature journal, and the offer is made to send it to Nature Communications – just launched, no impact factor as yet. We are tempted, and I will admit, mainly by the Nature brand. We are very confident it will get into a high impact factor pure physics journal, but still the brand is tantalising. Building such a OA brand will be difficult, when combined with the economics, but I think it should be possible. Who will take on that challenge though? And does our vested interest in the system give rise to a reluctance to try?

    • cromercrox says:

      @DundeePhysics – Best comment I’ve read all week.

    • Tom Hartley (@tom_hartley) says:

      I think the idea of brands is a bit out of date. Not to say they aren’t important now, but that in the future people will increasingly tap into articles directly from search and recommendation engines. If I want to find a good hotel I go to Trip Advisor, rather relying on a rather crude impression of an overall brand.

      OA fits better, in my view with post publication evaluation. With OA there is nothing to stop someone setting up their own list of recommended papers which (with sufficient expertise) would be akin to the contents page of Nature. Or we could each select a panel of experts whose opinions we trust and read the papers they recommend. F1000 already does something like this, and it works pretty well. The great thing is that the recommendations come from subject experts, but I’d be happy to read a list of expert recommendations prefiltered by e.g., Nature’s editors (and I do – for that is what Nature is, really). I am not sure how much I would pay for this, but I imagine the market could decide a fair rate.

      I think it would also be straightforward to produce personalised recommendations automatically using something like a Bayesian spam filter (which is how pubmed’s related articles system works). My own publications abstracts (and those of papers I read) provide a detailed profile of topics I am interested in, which can be used to identify titles and abstracts with related content. If these articles are also recommended by experts, or widely read by others, then I probably want to read them too.

      • I’d disagree that brands are out of date. I can discern for myself what is good, what is bad (at least in my field of expertise) and I have a reasonably good idea of how to find stuff I think will be interesting. This tends to have a reasonable correlation with the general community view of a journal – so in this case what journal the work is in matters little. However if you divorce the reading of journal papers from the other metrics that paper publications are used for (jobs, promotion, etc) then ‘brand’ is important. If I get a bunch of postdoc applications I will look at the publication list as one of the key points. It’s unlikely I will have read or will read all these papers, but a better impression is given if they are in the prestige journals. Now, as you say, you could set up an alternative system (a ‘like’ or recommendation system) as to how we could judge things, but i think we are a long way from that yet.

        Also post publication evaluation/commenting has been tried (including by Nature, I think) and the indication is that people are very unsure about making public criticisms about others work; equally many see little point.

        I think better systems should be possible, but until we find a way to eliminate the worship of prestige or set up our own OA prestige journals the status quo will be with us for some time.

        [I should set up my own Open Access Nature rival, how hard can it be ;-). Now, back to reading this week's Nature].

        • Tom Hartley (@tom_hartley) says:

          Well I think, again, that that will eventually change. As I say above I think we’ll soon have much better metrics so you’ll be able to judge postdoc applicants much better (by recommendations from independent people who’ve read their work, and by their readership).

          As you say unless scientists and academics act to change the current system, it will hang on for some time.

          I take that to mean that we should act – but it would help if we had resources from the top (government, funders).

          BTW, I expect that truly prestigious journals will survive in any event – as has been pointed out, they have a real, earned reputation that goes beyond the work that is submitted and reviewed by the academic subsidy. But the writing (IMHO) is on the wall for the others. It is only a matter of time.

          • Great post – great comments.
            @DundeePhysics: will the HHMI/Wellcome Trust/Max Planck Society OA journal (to be headed by Randy Schekman) do this – at least for the life sciences. Has significant potential to be the OA journal of choice.

            Limitation of the number of articles a la Nature/Science could ensure top quality. I could envisage a sister journal for the “rest” perhaps ultimately ending up with funder-mandated publishing in OA journals.

            Interesting times.

            (Declaration – am one of the editors for Biology Open from the Company of Biologists)

  24. cromercrox says:

    In my view (which I should emphasize is mine, not necessarily representative of NPG, my employer), and now I have calmed down a bit, I think that the publishing landscape has room for all sorts of modes of publication, for all kinds of people. It’s not an either-or market. For example, we’ve already seen that physicists are generally more inclined to use preprint servers than biologists.

    I suspect that open-access, free-to-air publication is more likely to be of use to small groups of specialists who know one another’s work fairly well – the kinds of people who would otherwise use small specialist journals, the kinds of journals that have regular circulations in the hundreds rather than the thousands, carry enormous subscriptions, and which libraries are most likely to cut.

    Here’s an analogy. If you are publishing a specialist book, or a book that will appeal to a small and identifiable niche market, it makes little sense these days to approach a conventional publisher. Better to publish it on a print-on-demand site as a downloadable file. That way it will get to your market at a fraction the price demanded by a conventional publisher, who has overheads to think about. I guess the same model would work – and, I guess, probably works already – for specialist titles with identifiable niche markets.

    There are all kinds of publishing models out there. Nature works on the traditional reader-pays, closed-access model, whereas with PLoS the author pays and it’s OA. At NPG we’ve been experimenting with mixed models (Nature Communications) and straight OA titles. There are other modes, too, such as free distribution to a closed circulation, such as members of a society, or a mailing list, with costs defrayed by advertising.

    • Frank says:

      Henry – I do agree. Trying to find a single model, to sweep everything away in the face of e brave new world, is futile. Some parts of the current system will continue to be useful, but none has an inalienable right to stay the same – not even Nature!

  25. Tom Hartley (@tom_hartley) says:

    @cromercrox I think you are about right. However, I think that the specialist/niche book analogy applies to pretty much all academic publishing. I think the very top-level discipline specific journals and perhaps highly selective general journals such as Nature and Science add value because they have the resources to promote the work to a wider audience than might otherwise find it, and the professional editorial and production staff perhaps can claim to add significant value. I am really not sure that’s true of the remaining 95% of titles which cater to the very long tail of the impact distribution, and where almost all the editorial work seems to be carried out by unpaid academics. In many cases these journals seem to have an interest in keeping too many prying eyes away from the research (since that is the only way it will maintain its value) – pretty much directly against the interests of the authors.

  26. Frank Norman says:

    In his talk on Thursday last week, at the Royal Society, Michael Nielsen pointed out that it had taken about 100 years for the idea of publishing results in a journal to become embedded in the culture of science. I think the ideas of open access/open science date back about 20 years to the beginnings of Arxiv (possibly further if you include the open source software movement).

    So we should not expect change to be overnight. Maybe it will be another 20 or 30 years before the world of scholarly publishing reaches a new, and more open, equilibrium.

  27. Tom Hartley says:

    I still struggle to understand how academic publishing is the way it is, or at least how it continues as it does. I can’t think of any parallel business model. I tried to imagine how it would seem if encountered for the first time by an alien civilisation, and this is the result, a rare foray into creative writing. Please feel free to comment in parable form!

    http://the-white-dot.posterous.com/a-parable

  28. Shu Ito (@shuito82) says:

    There are some really nice comments and ideas floating around here, centering around Stephen’s original post.

    As someone mentioned above, there are currently no “higher”-tiered OA journals. But that is based on the old impact factor measurement. Has anyone recalculated a journal impact factor based on article-level metrics? I suspect that because of accessibility, OA journals might score higher based on this new system. Would this then push researchers to consider publishing in OA journals, if it is more likely to be seen/cited? The current draw for top-tier journals like Nature or Science is the “brand” prestige that they offer, but if we can change the way the “brand” power is measured, would it then push researchers further towards the OA side?

    More towards the beginning of the comments, there was some discussion about post-publication review. Stephen points out (and I agree) that there are lots of lower quality publications out there that probably got rejected during the peer-review process. I don’t think we can get rid of the peer-review process. In fact, I don’t think we should. We need some measure of quality control. But post-publication review has its place too. It doesn’t necessarily stop a given “low” quality publication from becoming public or available, but it would certainly stop others from accessing it, reading it or citing it. Combined with changes in metrics, post-publication reviews could potentially be a powerful deterrent for researchers considering publishing half-baked stories. If publication after publication receives negative post-publication reviews, it could negatively affect a researcher’s reputation and credibility.

    All in all, changes do need to occur for science to move forward. Great post to start all this conversation though!

  29. Stephen says:

    Thanks all above for the great comments on impact factors — and all the other issues swirling around the tricky business of publication. I’m still processing…

    To pick up on the last point raised by Shu, which echoes @DundeePhysics, while there are no top-tier OA journals, they may not be so far off, PLoS Biology has an IF of about 13 which is very respectable. And I think some other titles from the PLoS stable aren’t so far off. Clearly that’s based on selection and perhaps these OA journals can only afford to be selective because of subsidy from PLoS ONE? Which takes us back to a familiar argument.

    And I agree with Henry that a mixed economy of publication is probably no bad thing — though I would like to see stronger support/enforcement of OA options on all papers funded from public/charitable money (wherever they are published).

    Frank — for sure, things are a-chanin’. But I can’t wait 20 or 30 years… ;-)

    Tom – I’ll leave the parables to you! Tickled that this thread has inspired a new form of commenting…

    • Frank says:

      The thing is, evolution will not be hurried. All we can do is to adjust the selection pressures. Since the 1970s the pressures on scholarly publishing have been largely commercial pressures and we have a system that prioritises profits.  Publishers say that their business is serving the needs of the scholarly community, so our needs are taken account of, but I think those needs have not been well articulated. Hence we have a system that delivers PDFs onto the desktop and that’s it. 

      We are now working out what we want scholarly publishing to achieve and putting in place some new pressures (funders’ open data policieies, microattribution, etc) to push the system that way. The report of last year’s NSF workshop, Changing the Conduct of Science in the Information Age summarizes some of the things we need to do.

      • Stephen says:

        Well, I think a little agitation from the scientific community might help to move things along a bit faster…

  30. Frank says:

    Phew! I’ve just read the whole post and comments, so a couple of overall comments.

    One of the problems is that journal publishing is many things. It is a cultural system (journals are communities) and there are different cultures in different subjects, influenced by the mechanics of research in different sciences and by the funding models. It is a business, and has become very complex in its business models. It also has technical components, providing the heart of digital science.

    It is hard to address all of these aspects at once, but they are interconnected.

    Richard Poynder has written a long but interesting account of how journal publishing got where it is today, and particularly how the Big Deal has affected the journals ecosystem. Librarians have been fretting about the “serials crisis” since the late 1970s. The crisis is that old, though many (most?) publishers are still happy to deny that a crisis has ever existed. Possible solutions only emerged as the Internet became widespread through the 1990s. Poynder says that publishers introduced big deals to solve the crisis, but they have ended up just as another part of the problem.

    For me the best thing about the last ten years is that academics and academies are now fully engaged – accepting that there is a crisis and thinking about their own roles in perpetuating the problems and ultimately forging a new model or models of publishing.

    • Stephen says:

      Frank — I certainly agree with you that the issues are complex and interconnected! I’m still trying to get my head around my own comment thread.

      Started reading Poynder’s report – very interesting background to the publishing industry but very long! Only about one third the way through and have bailed out to get on with something else. Anyone got an executive summary?

  31. Frank says:

    Oops- I forgot the closing tag!

  32. nico says:

    At the risk of sounding like an industry shill (but hopefully securing a salary raise), I would like to point at a few project that publishers are involved in.

    ORCID: once up and running, this could become not only a way of ensuring who published what, but then also to track who they cite, who they are cited by etc. Potentially one of the things that will partly displace the impact factor.

    Dryad and FigShare: data repositories, either for data associated with published papers (Dryad) or just any data (FigShare). Everything uploaded to FigShare is available Open Access and under Creative Commons licensing, I don’t know about Dryad.

    Others: Macmillan has recently spun out Digital Science, an “incubator” for new, web-based products to assist researchers. FigShare is one of the projects recently taken up by DS.

    Over at Nature we are always looking at ways to make our papers more useful, in the recent makeover for example the figure layout was changed to make them easily readable on screen. iPhone and iPad apps were released that use epub to make reading on a small screen easier (I understand an Android app is under development).

    Links:
    http://orcid.org/
    http://datadryad.org/
    http://figshare.com/
    http://www.digital-science.com/

    • Stephen says:

      Thanks of the links – will definitely explore. For sure this whole business is evolving. However some of these systems seem rather fragmented — why should data associated with a paper be in one place and the paper itself in another. Seems an unnecessary complication of the archive though I imagine smarter people than me have thought about this. One other concern with web-based solutions I guess is the question of longevity — would be a shame for any repository to disappear without someone else taking over.

      Good luck in the quest for a raise!

      • nico says:

        “why should data associated with a paper be in one place and the paper itself in another”: AFAICS it is because repositories tend to specialise in one kind of data. Hence we have PDB and UNIPROT on the protein side, GenBank on the genomic, etc. Now people try to set up “everything else” repositories, I don’t know how feasible that is but it is a commendable aim. That would be the ideal complement to the electronic lab book.

        “One other concern with web-based solutions I guess is the question of longevity”: this I am very concerned with. This week’s paper version of Nature will still be readable in 500+ years after a nuclear cataclysm, as long as the paper doesn’t get too damp. The web version though, all you need is power cut and…

        Although I sympathise with the Open Access model (I tried to look up a Nature paper this week end, but couldn’t remember my password…), and understand the argument that research funded by the public purse should be publically available, I believe most people who need and can understand papers can usually get them without too much hardship. Maybe everything should be OA, but I do not think that will happen.

        I am more in favour of scientists/publishers moving to Open Source software and standards. It’s all very well to have your .xls spreasheet available for all to see your calculations, if it is an old 95 version with obscure macros it probably won’t work nowadays. All those fancy next generation sequencing machines also often spit out proprietary formats that can be analysed only by their software. That is a way that is just as effective in locking down data as a paywall, maybe more effective in some cases.

        • Stephen says:

          Many have access but most? And for the public it is prohibitive — that’s the key point. I realise they are not the primary audience and there the condensed form of the scientific paper necessarily involves a degree of jargon. But I think that a move to OA would raise the profile of this audience and make scientists/authors think a bit more about them when writing. If your science is just for other scientists, are you really tackling the most important problems. (OK – that is a huge question and not entirely related to the main thrust of this debate). But it illustrates the potentially dynamic nature of changes in the publishing landscape. It had been raised by Kent Anderson and Andrew Maynard wrote a thought-provoking reply.

          Interesting position on open source…

          • nico says:

            I understand that $32 a pop for a Nature paper isn’t very cheap if you want the PDF, in fact it is probably cheaper to order an entire paper issue (http://nature.custhelp.com/app/answers/detail/a_id/679), although apparently the catalogue only goes back three years.

            There might a change overall in scholarly publishing, but as Henry pointed out there will always be room for different approaches. OA has made a lot of science much more accessible, and that has to be commended, but is the proverbial “man on the Clapham omnibus” more engaged with science because of it? I doubt that, although it can be argued those are still early days. I’d say it has made material more easily accessible to freelance bloggers or writers, and that is positive, but not a revolution yet.

            I cannot comment on whether these “one-off” purchases are priced too high, there are too many factors I am not privy to, such as how much of our income derives from one-off purchases vs subscriptions, how much it costs to maintain an online vendor system, etc. And if I knew I probably couldn’t tell you without killing you afterwards.

            • Stephen says:

              Is there anyone at Nature would care to offer a justification? Looks like ‘market forces’ to me — but alas the market can be rather brutish at times.

              But, tbh I don’t want to make this discussion all about Nature. I don’t want to give the impression that I’m picking on you guys; I certainly appreciate all the very thoughtful comments you and Henry have contributed!. Nature is a bit of a special case and I want to keep the discussion broad.

  33. Stephen says:

    Peter Coles, an astrophysicist from Cardiff, whose 2009 post on academic publishing I linked to in my post above has weighed into the debate again today with an articulate defence of the arXiv model.

    In it he cites a report by Ehsan Masood in Research Fortnight that says “UK universities, according to RLUK, spend almost £200 million on access to journals and databases—10 per cent of quality-related funding.”

    That seems to me an astonishing figure. Of course, it would be important to pick apart journal and database subscriptions, but I had no idea we were talking that kind of money.

    • Tom Hartley says:

      So by helping to sustain the system we’re sapping vital resources which could be sustaining junior scientists and giving them more secure careers. Are PIs really helping their PhDs and postdocs by publishing in high IF journals. Maybe. But not as clear cut as it first seems.

      If the £200M went straight from the funders into the pockets of the publishers’ shareholders, no one would put up with it. But it is almost as if researchers are laundering the money.

      As has been pointed out, some publishers and some publications provide a useful service and I am sure it can continue to be profitable, but some of the profits and charges seem massively out of line with the value they add to what is really (or ought to be) a public good.

      • Stephen says:

        Yep. Of course the costs are never going to be trivial — it’s important to spend appropriately to do a good job of publishing but that 10% figure would be a surprise to many I would wager.

  34. Laurence Cox says:

    As an outsider, someone who has worked in industry since gaining his PhD but is a member of three professional institutions (IoP, RAS and OSA), I think that the best approach is for the professional institutions to take back responsibility for publishing papers – in many cases this already occurs: the RAS publishes MNRAS and OSA publishes JOSA and Applied Optics amongst others.

    In the old days (1970s and early 1980s) both JOSA and Applied Optics came free with OSA membership. If I remember correctly, MNRAS also did but only for Junior Members (below 26). We need to look at what the annual subscription cost would be if all members of a professional institution received a electronic copy of one journal (or equivalent access) as part of their membership subscription.

    Secondly, the professional institutions are in the ideal position to know who are suitable referees for papers. There needs to be some payback for refereeing: bearing in mind that referees often make significant improvements to papers, yet can only be cited by the author thanking the anonymous referee. Refereeing a number of papers (TBD) should be considered as equivalent to a sole authorship paper for the purpose of counting a researcher’s publications.

    Another issue that the professional institutions would have have to deal with is the dichotomy between the interests of their members in academia and in industry. For the latter, attaining a CEng, CPhys, or CSci qualification is of much more importance than publication record, which in any case is often compromised by considerations of commercial confidentiality.

    • Stephen says:

      Thanks for chipping in Laurence — good to have an industry perspective. I guess the OA model would serve businesses well since they save on subscriptions (as readers), though since they’re not really ‘the public’, I wonder if it would be appropriate to expect industry to contribute in some way to access knowledge that they can put to good use. Or maybe the wider benefit to UK plc is more important?

  35. Pingback: Commodification, the Academic Journal Racket and the Digital Commons « The Disorder Of Things

  36. nico says:

    Interesting report from the Scholarly Kitchen about PLoS turning a profit for the first time: http://scholarlykitchen.sspnet.org/2011/09/15/plos-2010-progress-update-pondering-the-implications-of-a-watershed-year/

  37. Stephen says:

    And this just in from RCUK – a working group comprising funders, academics, publishers and librarians has been set up to “to examine how UK-funded research findings can be made more accessible.”

    Good that people are now sitting around the table to thrash this thing out!

    Report out in spring next year…

  38. Stephen says:

    Haven’t forgotten this issue in all the excitement of finishing my film

    …here is a good piece on OA publishing in the New York Times

  39. Stephen says:

    FYI: Found out this week that my university — Imperial College — is supporting the stance taken by Research Libraries UK to try to resolve the problem of unaffordable periodical subscriptions currently being demanded by Elsevier and Wiley.

    • Stephen says:

      Yes – although looks like they are still buying into the impact factor which, as we all know, is all too easily abused.

  40. Pingback: Why I chose to decline an invitation to review by Elsevier | Reciprocal Space

  41. Pingback: An Open Letter to Elsevier | Reciprocal Space

  42. Pingback: The publishing revolution | Naturally Selected

  43. Rc tanks says:

    Monbiot should not have strirred up the situation like that. It is only going to lead to more agitation.