I had an argument with my colleague in the tea-room the other day. Gratifyingly, I learned he had been reading my blogposts on the subject of open access, but it soon became clear he did not entirely share my enthusiasm for the topic. Specifically, he criticised open access journals such as PLoS ONE both for their lack of sub-editorial services and for creating a home for poor quality science.
This got my goat, not least because I had made my first submission to PLoS ONE just the day before. We spent some time arguing back and forth and my colleague was kind enough agree to let me lay out the dispute in a blog post. I want to do so because, despite the evident variety of opinion within the blogosphere, it can be easy for the like-minded to coalesce into groups where positions are not so rigorously tested. So please see what you think of my case (since amplified by further reading) and feel free to take issue.
The first charge sticks only partially. There is no copy-editing of articles in PLoS ONE — this is one of the factors that keeps the OA charge to $1350 — but it is nevertheless the case that the articles accepted by the journal are formatted and certainly look like the real thing. Here is an example, selected at random. It may well be that authors miss corrections that a sub-editor would catch but authors can mitigate the worst offences simply by asking co-workers unfamiliar with the paper to proof-read for them. My colleague was prepared to pay a couple of thousand dollars in page charges for formatting and copy-editing (not including open access fee). Perhaps I am a cheap-skate but to me this seems ridiculously expensive.
(As I side-issue I think one of the important advantages of open access is that, by moving all the charges to authors, the real cost of organising and disseminating the scientific literature becomes visible. Authors at universities rarely see or care about library subscription charges, but the transparency of the OA model provides useful downward pressure on the costs of publishing. Let’s not forget that most of those costs are met from public or charitable purses.)
The more serious accusation, of opening the sluice-gates to a torrent of sub-standard science, bears closer consideration because of the unusual threshold for publication that PLoS ONE operates. Unlike most journals, they do not consider the importance of a paper’s findings when judging whether to accept it:
“PLoS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound. Judgments about the importance of any particular paper are then made after publication by the readership (who are the most qualified to determine what is of interest to them).”
My colleague was troubled by this statement, seeing it as a licence to pollute the literature with low-grade science. There may be a modicum of truth in this view. I suspect it is widely shared, stirred up in part by reports of the reprehensible practices of other OA publishers such as Bentham Science, but I still want to take issue.
First, the publication of mediocre science is by no means the exclusive domain of open access journals. There is an awful lot of it out there, as anyone involved in peer review will know. The scale of the problem is terrifying: half of the 50 million scientific papers estimated to have been published since 1665 were published in the last 25 years. Worse still, the fraction of the literature that is not cited within five years of publication has increased from 55 to 59% in the past 20 years. The incredible growth of the scientific literature, much of it deservedly unread alas, is due partly to the rise in the population of working scientists but probably also, as others have argued, to the excessively competitive nature of science and to increased demands to quantify output.
It’s hard to see that open access journals are at the root of this problem, though they are certainly part of the ecosystem. But even if they are adding to ongoing difficulties within science — issues that need to be addressed separately — I would contend that this a price worth paying for increasing the accessibility of the scientific literature.
Second, it would be quite wrong to assume that PLoS ONE will publish anything. In fact the journal rejects about 30%* of submissions (according to Cameron Neylon), half of which end up being published elsewhere. So, although PLoS ONE is not judging work on ‘impact’ and although the journal editors and reviewers have to manage high volumes of submissions and are not in a position to apply the rigour that submissions to Nature or Science would be subjected to, the criterion that manuscripts have to report work that is ‘technically sound’ still appears to be providing a useful filter, at least on average: the Impact Factor for PLoS ONE is a respectable 4.4.
That filter might yet be enhanced as a result of changing attitudes to scientific publishing. The Elsevier boycott has focused much-needed attention on both the issue of open access and the very high profit-margins of commercial publishers. Journals from the PLoS stable — and other open access publishers — may come to be seen as more legitimate recipients of the freely-given efforts of editors and reviewers. Any increase in the pool of talent for these roles is likely to sharpen the peer review process at PLoS ONE.
In any case readers provide their own filters. These days, few would browse the tables of contents of all the journals likely to interest them; no-one has the time to scan that much material and in any case the tools for automatic sifting (e.g. via PubMed searches) work far better. An informed reader can quickly assess search results to see if a paper’s findings are a sufficient buttress for the conclusions trumpeted in the abstract. If not, one quickly moves on.
All of this is not to argue that journals like PLoS ONE represent the entire future of academic publishing. Bob O’Hara has sounded a cautionary note about the future financial viability of the PLoS operation, arguing that once traditional publishers jump on the band-wagon, PLoS will find it harder to tout for business. I don’t buy that since I think PLoS has helped to establish an ethos that chimes well with publicly-minded scientists.
But these are certainly interesting, if argumentative, times.
*Corrected on 4-4-12 from an initially quoted figure of 40% in response to a comment from Peter Binfield (who runs PLoS ONE)
I like open access but I don’t like this statement in the PlosOne editorial process page:
The academic editor can …
“conduct the peer review themselves, based on their own knowledge and expertise”
http://www.plosone.org/static/guidelines.action#editorial
ie. a paper can be published in PlosOne based on one editor liking it and choosing to bypass peer review. I don’t know how often that happens, but it worries me.
Whats wrong with that? This is how Nature/Science etc operates where the editor without being expert in the field decides whether the MS should be reviewed or not. As Stephen pointed out there many high profile publications that do not receive any citation despite making it to top class journals.
When business and numbers are integral to academics I guess this how it would look like. Now we better don’t regret as each us should be held responsible for what we deal today.
Your colleague thinks of page fees as paying for professional copy-editing services? S/he still needs to proofread his/her own article. “My colleague was prepared to pay a couple of thousand dollars in page charges for formatting and copy-editing (not including open access fee)”. That strikes me as way sloppier than anything at PLoS ONE.
In response to anonymous, some high profile journals sometimes accept papers on the say of an expert editor, so te PLoS policy isn’t unusual.
I wonder if anyone has calculated the cost of getting a professional scientific sub-editor to go over a paper? Is it really in the region of $1000?
I agree the practice is not the reserve of PLoS ONE. I once got a paper into J. Biol. Chem. after only a single editor had reviewed it. I confess I did not insist on more reviewing but it did make me feel uncomfortable. I’d prefer at least two people to be involved in reviewing (even if one of them is an editor).
I don’t imagine for a moment that decent copy-editing costs that much but of course the journal is also charging for infrastructure (and a hefty 35% profit margin in the case of Elsevier titles).
Yes, I’ve had single referee reports on stuff we’d sent to J Biol Chem, although more recently it has always been two reports.
I guess the difference might be when the editor feels au fait enough with the area of the work to make an expert judgement on it?
In general, my experience has been that the old-style top-end-of-specialist-category run-by-learned-societies journals, like J Physiol, Brit J Pharmacol and Biochem J, have the most rigorous (as in ‘three referees’, sweep and detail, picky editors) peer review.
As doubtless someone has already said, the closer you get in the journal hierarchy to the GlamRagz, the more likely it is that the primary process is ‘editorial triage’. Though, once past that – which I always feel is mostly an ‘impact’ barrier – the peer review is not that noticeably different from anywhere else.
Jmarlesw: you can find out fairly easily online how much it costs to get a scientific paper copyedited. PLoS has a list of scientific editing services, and some other publishers have similar lists. One editing service that I work as a freelance editor for, Edanz Editing, charges 20 USD per 250-word page for copyediting, or 25 USD for more substantive editing. The papers I’ve edited for them average at 17 pages of this length, which would make the fee 340 or 425 USD.
So a figure of 1000 USD seems only a little higher than I’d expect, and particularly long or badly written papers could easily cost that much to copyedit. Of course the costs of copyediting depend on how high a standard you want. Some journals, certainly not all, also proofread their papers after the author has checked them.
If you’re a non-native English speaker submitting to a journal that doesn’t copyedit I would highly recommend using an editor (though I am of course biased). Many freelance scientific editors (like me) offer copyediting directly to researchers – you can find them on the directories of organizations such as the UK Society for Editors and Proofreaders (SfEP) or the US Editorial Freelancers Association (EFA) and ask for a quote. (I won’t add links in case this comment gets junked as a result, but the websites are easy to find.)
Thanks for the info and the breakdown Anna. I agree that paying for proofing is a small price to pay to improve copy, especially for non-native English speakers.
Stephen,
I am with you! I have been an academic editor for about 3 years with PLoS One, and I can tell you that while there is probably a great deal of variability between the standards of individual academic editors, I do think that this is NOT a dumping ground for bad science papers. Many manuscripts are rejected, and I find that the reviewers in my field to whom I send manuscripts are often quite finicky about ensuring proper controls and interpretations.
I would also like to note that while potential impact is not taken into consideration, this does NOT mean that there is a license to publish work that others have already published, or superficial studies, or studies that do not lead to specific conclusions.
Having said that, I have not chosen PLoS ONE (yet) as a journal to publish work from my own laboratory, simply because in recent years we have been fortunate enough to have findings that generally are of “impact.” But I certainly would not be embarrassed to publish in PLoS ONE in the future.
Good to hear, Steve, though nonetheless slightly odd that you have never published in a journal where you serve as an editor.
But it sounds like things are changing. As further impetus, let me share a couple of things that were pointed out to me via Twitter since I put this post up.
First, Heather Piwowar (@researchremix) informed me that 83% of the papers published in PLoS ONE in 2010 have been cited (detail here.
Second, Bjorn Brembs (@brembs) directed me to a h-index based ranking of journals that puts arXiv up near the top with Nature and Science. Notably, it also supports Heather’s data on the ‘impact’ of PLoS ONE, which comes out as the most highly ranked PLoS journal, in spite of acceptance criteria that specifically ignore impact.
It’s an interesting observation that PLoS ONE has a higher h-index than PLoS Biology. But given how the index is calculated, even if highly cited articles are rare in PLoS ONE, my intuition is that the sheer volume of articles will generally lead to a high index.
I think you’re on to something there… It’s not a perfect measure by any means.
I’d love someone to do the maths on this but my intuition is with you on this. I think it will also apply to impact factors. Assuming an equal distribution of citations, a larger journal will sample more of the long tail, and because the IF is an arithmetic mean this will have a disproportionately large effect. Is that a good thing, a bad thing, or just another indication that these kinds of metrics are not very helpful?
Your intuition’s wrong (I discussed this with Bjorn on twitter a couple of weeks ago): it’s basic sampling theory. Yes, a larger journal samples more of the tail, but the weight of each paper becomes less. The only way it wouldn’t work would be if the sampling variance was infinite.
Good post. PLOS journals are as good as any other in my opinion. For In the case of PLOS One I admire particularly their policy of publishing good work regardless of whether the editors perceive it as trendy. It’s a good antidote to the absurd belief that Nature publishes papers of general interest.
The data seem to be bearing that out – see links in my response to Steve Caplan above.
The current system of peer-reviewed journals is altogether very flawed. The authors do the hard work of doing the experiment and writing the paper and then going through all the trouble of getting it published. Those interested in the reading the paper has to pay a ridiculous subscription charge to get to it. And at the end of the day, the journals make millions just formatting, laying them out and sending a few emails. This just cannot be right.
You’re correct that “this just cannot be right”, but that’s because your characterisation of a journal editor/publisher’s work as “just formatting, laying them out and sending a few emails” is a gross underestimate of what they really do.
I work for a learned-society publisher (my views are my own not my company’s, which is why I’m not giving more details), and although I’m not directly involved in journal editing, I’ve spoken often and at length with colleagues who are. As others have written elsewhere in the thread, their work on an individual paper usually begins with the editor deciding whether the paper is good enough to get checked by experts. If it is, he or she then needs to decide which researchers are best placed to carry out the review; this requires both scientific and personal knowledge, e.g. knowing who in the field are Dr X’s best mates and worst enemies.
After reviewers have been agreed on, it’s up to the editor or the journal’s administrative staff to make sure they follow through. I suspect this is the bit you dismiss as “sending a few emails”, but if you’ve ever tried to extract information from a busy and uncommunicative colleague – or waited impatiently for your own paper to be reviewed – you will know that this can be a nontrivial matter. Automated reminder emails are easy to send, but also easy to ignore; quite often, you need a human to get involved.
Once the reviews are in, the journal editor may need to adjudicate between conflicting opinions. This again requires specialist knowledge of the field, and also a good network of contacts, e.g. members of the journal’s editorial board, to provide advice if needed. If the conclusion is that more work is required before the paper can be accepted, at least part of the above process is repeated.
If the final decision is to accept the paper, then yes, it is copy-edited and formatted. Others on this thread have already noted that good copy-editing costs real money, so I will just add that although requiring authors to use standard templates makes formatting easier, there is often still a lot of work to be done. No journal editor worth his or her salt is going to reject a paper just because the author’s TeX skills aren’t up to coping with, say, multi-line equations, and anyone who’s ever spent hours wrestling with recalcitrant code will know that fixing formatting bugs is not always a trivial task.
So far, all I’ve covered is just the day-to-day stuff – which, by the way, you can multiply by dozens or even hundreds of papers per day for a really popular journal. In addition to this, senior editors and publishers also do a lot of work behind the scenes to build the reputations of their journals, attending conferences, keeping up with the field, trying to attract the best papers by offering more efficient or better services (e.g. getting the publisher’s marketing staff to help promote breakthroughs and enhance “discoverability”) and so on.
All of this stuff costs money, but the main expense, as I understand it, is salaries. The people involved are highly trained, mostly to PhD level or beyond, so to attract entry-level talent you need to pay them at least as well as a postdoc. The resulting salaries are hardly in 1% or even 10% territory, but they are well above the UK average wage of around £25k. Publishers could potentially save money by exporting those jobs overseas (many already do this for low-level copy-editing and formatting), but to some extent you get what you pay for.
In addition to editors, you also need support staff, notably IT. As well as building and maintaining a paper download and submission system, a publisher’s IT department is responsible for the complex (sometimes bespoke) systems for managing the peer review process. I occasionally see comments about how publishing ought to be so much cheaper now that so much of it is electronic, and to some extent this is true, but paper and lorry drivers are cheap whereas computer kit and IT staff are expensive, so the money saved is not as much as one might think.
So your open-access fee is actually paying for quite a lot. In addition to the one-off costs of copy-editing/formatting and peer review, there’s also the cost of making those papers available in perpetuity (usually not high, but there’s always the chance that a disruptive new technology will come along, like archive digitization did a decade ago), and subsidizing the review process for papers that got rejected. There are also some industry-wide projects to improve data metrics; I know of one multi-million pound effort at disambiguation (making sure the J Smiths and L Chens of the world can be told apart, and that when J Smith gets married and becomes J Jones, her papers are still somehow “tagged” as being hers), but I am sure there are others.
I could make a few other points about funding of OA publishing, but I have intruded on Prof. Curry’s blogspace for long enough. I happen to believe that publishers have been really atrocious at communicating what they do, though, so I hope this has helped a little.
I agree that the process is slightly more complicated that just sending a few emails. However the fact remains that the publishers make enormous profit off the back of publicly funded research. Seems like the fees in that case more than pay for the workload and expenses you suggest. Just a thought…..
“Second, it would be quite wrong to assume that PLoS ONE will publish anything. In fact the journal rejects about 40% of submissions (according to Cameron Neylon), half of which end up being published elsewhere.”
You might be interested in looking in the comments section of this blog, where I and Pete Binfield, publisher of PLoS ONE, provided information and clarified what happens to the submissions rejected by PLoS ONE. Details are also given of the reasons why, other than being unsound, submissions might be rejected/not end up being published by PLoS ONE.
Thanks Irene – yours and Pete Binfield’s comments on that blog are very useful.
Also – should I correct my figures on rejection/re-acceptance rates at PLoS ONE? Your comments indicates a rejection rate of 31%, of which about 44% are then published elsewhere.
I’m not sure I entirely follow the recommendations for improvement of peer review that are outlined in the blog post itself. Is it about turning every published paper into a wikipedia article?
Thanks Stephen. Interestingly, just last week PLoS Computational Biology moved to an interesting model where in addition to publishing Topic Pages in the journal (the ‘static’ versions) they are posting them to Wikipedia, where they will become ‘living documents’ that will be updated and enhanced.
I think the opportunities for innovation in research publication are enormous and the field is wide open – for new initiatives pre, during and post publication, especially the latter as publishing models are changing. I very much like the PLoS ONE model, where all sound/scientifically rigorous work is published. It’s not right that sound work should need to go from journal to journal, being reviewed multiple times, often not being accepted on the grounds of insufficient interest or novelty. Sound work needs to be out there and available to other researchers to use and build on, with researchers moving on with their own work instead of chasing publication. But there needs to be sorting, evaluation, and signposting post-publication – I think this might be what David Rosenthal meant in his comment to me that both humans and search engines need help to understand the quality of the published research they are reading. But that’s where currently there is a gaping hole.
I’m reluctant to comment on how best to express rejection/acceptance rates for PLoS ONE as I’m not with PLoS – you’ll just need to make a judgement based on the info in the HofC inquiry responses and what Pete Binfield has posted. You might also be interested in the slides with the results of the 2010 PLoS author survey (can’t find any for 2011), which include data for PLoS ONE.
sorry, links above don’t work, here they are
http://www.ploscompbiol.org/article/info:doi%2F10.1371%2Fjournal.pcbi.1002446
http://www.slideshare.net/MarkPatterson/p-lo-s-author-research-2010-6638756
Thanks – fixed those links for you.
Yes, just to clarify. Irene is pointing at figures that Mark Patterson gave in evidence to the parliamentary enquiry on peer review a few years back. The rejection rate for PLoS ONE has actually gone up a little since then, bringing it closer to the 40% figure that I quoted from memory and I believe that recent author surveys give a roughly consistent figure for number of papers eventually published elsewhere, so slightly less than half.
Actually Cameron, your data is wrong – the weakness of not yet being fully inside our inner circle! 🙂
Although the rejection rate has changed a little since that evidence, as you note, the current acceptance rate is still around 68%-69%
Pete
Just to clarify – the peer review inquiry at which Mark Patterson gave this evidence took place May-June last year (2011). The report ‘Peer review in scientific publications’ was published 28 July 2011. For anyone interested, you can access it (free, html and pdf) via
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/85602.htm
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
The main part of the report is on pp 1-95, the oral evidence (all the Q and A) and the written submissions from the witnesses, including the supplemental evidence sent in response to requests for further information from the Committee after the oral evidence sessions, are on the final 148 pages.
Government and RCUK responses
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/1535/153502.htm
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/1535/1535.pdf
“My colleague was prepared to pay a couple of thousand dollars in page charges for formatting and copy-editing”
Your colleague still can, using one of the several third-party scientific manuscript copy-editing companies available. (In fact, if the going rate is a couple of thousand dollars per article, your colleague could pay me to do it.) I wouldn’t be surprised if PLoS already tell authors of scientifically sound but badly written articles to do exactly that.
I’ll pass that on…
I’m increasingly sending my work to PLOS One as first choice of journal. I’m old enough not to care about my CV and just want to make sure that my work gets out there as efficiently as possible and that people can read it right away. Also, I’m Wellcome funded and obliged to make my work Open Access. I am impressed by the process: it is usually pretty quick once accepted. It’s by no means an easy ride, though. I have always had at least one reviewer in addition to the editor, and usually two, and some have been very picky.
The major turn-off is that it’s a pain getting figures to a standard they journal accepts – though they do give detailed instructions which work if you have Photoshop – Last Saturday was a case in point: 3 hr spent tweaking figures that looked fine in Matlab or .gif but needed to be converted to high resolution .tif with specific dimensions. Although they don’t edit your prose, they do a check of your ms for compliance with their style which can be tedious – once I had to fiddle with all the little square brackets for ages and thought it would have been easier for the journal staff to do it than to explain it to me. But having said that, I’m finding it is increasingly the case that non-OA journals also expect you to act as graphic designer and copyeditor as well as author, so I just have to bite the bullet and do this.
A very nice feature of PLOS One is that you can check the metrics on your paper after publication and see how many people have viewed or downloaded the paper. In time, I suspect this kind of metric will be used more than journal impact factor – it’s certainly a better indicator of impact for the individual paper.
Many thanks for sharing that — I’m sure it will be very useful for others who may have been hesitating on the threshold of PLoS ONE.
I share your frustration with the journal’s approach to figures. Mine failed to pass muster on submission because the tiffs I submitted included a colour correction profile that was disallowed. The instructions that popped up were to use Photoshop to remove the profile but I don’t have a copy of that program! Fortunately, half-an-hour of exploration was enough to discover that Pixelmator (a much cheaper image-editing package — Mac only) could achieve the same thing. I have written myself instructions on what I did so that I won’t get into the same problem next time around.
I just set up a little script that produces my figures to the appropriate format. Do it once and you’re done [took about 5 minutes]. In saying that, I work with R for my figures, where such things are simple if you are used to it!
It may be in the best long-term interest of PLOS to put small example scripts in commonly used data analysis packages [anything that are used to generate figures from data using scripts] as a link within the formatting instructions. Avoiding many people repeating the same 3 hour process needlessly will remove another percieved barrier to people adopting the no copy-editing process. This, of course, won’t help with figures that are produced by other processes, but it would be a start.
Some OA journals like the one you describe keep the flood gates ever so slightly ajar to make them financially viable. Many scientific papers are good in these journals. However, just as free newspapers (such as those received on urban transport systems….mm) report current and ‘truthful’ news, they are required to punctuate their pages with a litany of useless adverts to fund themselves. There is also much ’cut and paste’ and loads of typos – as a whole entity it is a dreadful read !
Eh? Please show your working.
And secondly, all broadsheet, Berliner and tabloid format newspapers contain a “litany of useless adverts” and more cut & paste than you might imagine (regurgitated press releases). Paper and online versions. Free or costly.
Stephen, once again I think you’ve again managed to outline the relevant, important points. I’ve never submitted to PLoS One, but am coming very close. Partly due to frustration with traditional ‘impact’ judgements, but also due to a desire to get my work read by as wide an audience as possible and an appreciation that this should be the way forward, via PLoS or any other publisher.
Well, as I said the other day, I don’t think the metaphor of the free commuter newspaper stands up. As we have seen, although the rejection rate is nothing like the top tier journals, there is still some valuable sifting going on. Worth also bearing in mind the flaws and excessive pressures that afflict the highest impact journals ( which themselves sometimes publish scientific results that turn put not to be valid).
I don’t know what the best measure is for assessing the content of kljournals like PLoS ONE, but I don’t yet see the evidence that it is as bad as you contend. No-one reads it ‘as a whole’; these days people are very good at sifting.
I for one am very aware of university subscriptions. My university (Bath) has outstanding online journal access, yet those costs PLUS new books are less than the cost of 2 PLoS articles per academic per year. The PLoS model for funding publishing does NOT benefit universities, it attacks them. So who benefits? Casual readers, but is that worth undermining our society’s research engines? Small companies? Maybe like with health insurance it would be better to support these through some aggregate system?
And yes, I’m very worried about the moral hazard of buying publications. Maybe I’m just unlucky, but when I’ve seen authors do end runs around published criticism of their approach it has mostly been in PLoS ONE.
You raise some interesting points Joanna. I don’t have the link to hand but research commissioned by the Wellcome Trust purported to show that the aggregate cost of OA publishing was lower than for the traditional subscription model. The problem with moving to full OA is that the support for OA charges and for library subscriptions come from different sources. That requires a fix at a pretty high level.
And it is true that research intensive universities would probably end up paying more but that imbalance could be solved by matching funds for publication costs to research income. In effect, the Wellcome Trust probably already does this. So I don’t see a long term problem, provided that the finding streams are sorted out properly.
The question of whether an author pays model leads to more fraudulent behaviour is an interesting one. But I wonder if it is over-stated. Bentham, which got caught out (see link in my post) has surely seen its reputation plummet as a result (not to mention their incessant and irksome spamming!) I suspect the editors will keep PLoS ONE on the straight and narrow.
Too sanguine?
Richard, thanks for the post (not so sure about the title though…). To answer a couple of the questions that have come up:
We do have an approx 69% Acceptance rate and an earlier Author survey did find that of the 31% who were Rejected, approx 44% went on to be published elsewhere. As I mentioned in an earlier thread on a different post (http://blog.dshr.org/2012/03/what-is-peer-review-for.html), we are in the middle of a research project to look into this, and an early result from that project is that the number is lower than 44% (perhaps more like 25%). The reason for the difference (we believe) is because the 44% was self reported by rejected authors (who may have been more than averagely motivated to respond to our survey because they wanted to “tell us something”). So it may well be that 25%-30% is the more accurate number (we are still pending the final results).
As to creating a home for poor quality science, hopefully the comments above have addressed this. However, we are a home for all science. We do not believe we are publishing science which would not have been published somewhere else in the literature. Therefore, instead of ‘creating’ poor quality papers, we are instead speeding the overall publication process for all science, and reducing a lot of the wasted effort that normally goes into that process.
As to our policy that an Academic Editor can make a decision on a paper without external review. This ability is only permitted when an Academic Editor believes they are sufficiently expert on the topic of the paper to make their own decision as to whether or not it meets our criteria. In reality, only a very small percentage of Accepted papers are decided upon via this route (less than 5%) and on average, all our papers receive about 1.8 external reviews (in addition to the Academic Editor themselves). The interesting thing with this criticism is that the vast majority of journals in the world also allow this to happen (as noted above) but as a general rule they are not transparent about that fact (unlike PLoS ONE).
Many thanks for the informative comment, Peter. Just one question for now: who’s Richard? 😉
Argg. Two different threads going on in my head!
Will be curious to see what your colleague makes of the post+comments
No worries. I believe m’learned colleague has already stepped into the fray as ‘A.N. Colleague’ above.
Moderate apologies for the slightly provocative title – a trick taught to me by a journalist friend. 😉 If it’s any consolation, it could have been worse…
I was quite proud and pleased with my PLoS One acceptance recently. Our paper was too interdisciplinary to please elsewhere – we tried – & it took all that time to convince my coauthors.Like Deevybee, I like the ease of visibility and the metrics. For me, the review process seemed tough but very fair and that isn’t my first paper by a long shot. I will choose it again if possible.
Good to have this further testimony of the quality of the review process.
I guess it would be useful to hear from complainants about examples of poor papers that have made the cut…
In which journals? I think we’d need a rigorous comparison before we could judge if PLoS One (or other OA alternatives) were any better or worse than the traditional publishers. Given the higher submission load PLoS One now has, this would also have to be taken into account.
But I can easily give you a list of Science, Nature and PNAS papers that are flawed, if you like…
I suspect the highish impact factor for plos one is due to the automatic email you get when your paper is rejected from one of the other plos titles.
The costs for these other OA titles are below:
PLoS Biology US$2900
PLoS Medicine US$2900
PLoS Computational Biology US$2250
PLoS Genetics US$2250
PLoS Pathogens US$2250
PLoS Neglected Tropical Diseases US$2250
PLoS ONE US$1350
Notice a correlation with impact factor? What is the true cost if OA — few thousand dollars for NO copy editing. Is this profiteering for the other plos or are plos one playing the numbers game?If the later then standards must and do suffer.
Forgive my typos — I need a copy editor
I think the point about the possible detrimental effect of lack of copy editing is taken, though perhaps over-stated. It doesn’t seem likely that the cost differential above is exclusively due to copy-editing costs. I suspect the fixes (either through rigorous colleagues or 3rd-party copy-editors) are much cheaper.
Worth noting also that PLoS charges for OA are lower than Elsevier’s standard fee ($3000) and much lower than those of its Cell Press titles ($5000).
IIRC the PLoS titles other than PLoS One do offer copy editing and other ‘added value’ services, for example, PLoS Biology often runs ‘popular’ commentaries on the main research articles in each issue. They also produce a hard (paper) copy of each journal issue. This extra work might help explain some of the price difference.
Typos can be forgiven. Confusing correlation with causation, though…
I’d note that there is also a correlation with the age of the journal…not sure that that is actually any more significant.
But to get to the substantive issues – most of these other journals run with high rejection rates, PLoS Biology is rejecting over 90% of papers I think, and managing that peer review process costs money, alongside other things like copy editing. Actually if you calculate a cost per paper reviewed then PLoS Biology is a bargain (~$250) compared to PLoS ONE (~$1000). The truth of the matter is that PLoS ONE subsidises the high rejection rate of PLoS Biology and Medicine.
Second the majority (and I think the vast majority, Pete can probably give the correct numbers) of papers in PLoS ONE are not direct transfers from other PLoS journals. As a (soon to be ex- see below) Academic Editor I’d love it if there were more of these because there would be a nice set of existing reviews that we could take and make a quick decision. I think I’ve seen one transfer amongst all the papers I edited (and that wasn’t sent for review at PB).
Finally I think you’re making a category error when talking about “standards” for two reasons. Firstly there is no absolute “standard”. It is easy to multiply examples of papers that were neglected for years or decades because they weren’t appreciated, or their time had not yet come. But more importantly your standards may not match to what I need. This runs the risk of people’s opinion on importance blocking the access of others to information they need.
I want to know that the biophysics of exactly the protein I’m looking for has been studied and not just some homologue, or that someone has actually tried this clever method I think I’ve invented and it doesn’t work, or that someone has published something I can use as calibration data, or someone has done this in just precisely *those* buffer conditions. These are not random examples, they are things I wanted to know at some point in the past – where I knew (or worse, found out later) that people had done things but not bothered to publish them. If the price of that is that there will be a bunch of things published that I will never look at or even see, then that’s a price I’m more than willing to pay. Do we need better tools to find the things that are relevant to our needs, right now, from the mass of stuff out there? Yes. Is the way to build those to reduce the amount of information available? Absolutely not.
[Full Disclosure: I don’t yet but I will be working for PLoS from July, so I have stepped down as an Academic Editor and have declined referee requests for the past few months. I think the fact that without even discussing this, everyone involved assumed this would be the case is an indication of how seriously the division between charging and editorial selection is taking at PLoS]
My personal scientific discoveries, the ones I am most proud of, have often evolved from ad-hoc browsing of journals. In the old days this might have been a weekly stroll around a library and reading an article that was unrelated to my interests, but caught my eye. If Plos One existed back then I’m not sure it would have succeeded in this respect. We can automatically search all sorts of stuff now, I hear you say. In my humble opinion (and personal experience), hunting for scientific inspiration by ‘googling’ specific terms does not stimulate game-changing ideas, but leads to incremental step. Stumbling accidentally across a new idea, even in this digital world, is important and requires the highest standards of presentation, which is worth paying a little extra for (agreed perhaps not as much as is presently charged )
I absolutely agree with this – one of the most exciting challenges in the digital world is the one of “recreating serendipity”. I would counter that I have a great advantage though – I am not restricted to the serendipity of things occurring in the same table of contents but can bring together things from many journals and many spaces into a space where there is the opportunity for interesting patterns to form. And *I* get to choose how to filter what comes into that space rather than relying on some anonymous editors and peer reviewers.
I find this really exciting – because if we can get it right (and I would accept there are limited examples where this has been done well) we can combine both serendipity and directed focus. So I might choose to combine a close examination of your papers with research from another specific area, or perhaps with Keats, and I can experiment with what works for me in creating those moments.
Twitter can actually be used in this way somewhat effectively – but requires very careful choices about who you follow. FlipBoard on the iPad is a good example of a visual system that might work well to recreate that happy browsing accident experience of two things appearing side by side. But there is much work to be done here on making this work well I would agree.
What Cameron said. 😉
I agree there is great untapped potential here — and in the burgeoning social media side of science. What we need is a stumbleupon for science.
Sorry to be anonymous but as with A.P.E above these are my views and not those of my current or former employers.
I don’t have data to support this but I have worked on both ‘high tier’ high rejection rate journals and ‘mega-journals’ and I have not seen any difference in the quality of the peer review. It is even possible that the ‘mega-journals’ received better reviewers comments.
The fact is that peer reviewers (and the Academic Editors on journals) are conscientious. They don’t want the literature to contain poor science. With a ‘high-tier’ journal they might be sent a paper that they don’t find very interesting and will write a short review just explaining why they don’t see the results as significant; it isn’t worth writing more as they believe the paper is going to be rejected. If they get the same paper from a ‘mega-journal’ they expect that it will be published despite being not very interesting (in their opinion). As a consequence they will go more carefully through the details of the paper finding inconsistencies and giving advice to the authors on how to improve their paper through revision, it is worth the time it takes because the paper will likely be published.
This is anecdote I know. I can’t think of a way to come up with rigorous data. But I have been very surprised by the general quality of referee reports on mega-journals as compared to top-tiers. Very pleasantly surprised.
Completely agree. I have published one paper recently in PLOSone was surprised by the quality of the review. Interestingly, a colleague of mine used to criticize PlosOne. Then just the other month she published for the first time PlosOne. And now she has good things to say. Surprise, surprise and surprise!!!. I guess we will see more converts like her.
Against the Grain is holding a poll on the question: “Does the gold open-access model of publishing threaten academic integrity with minimal peer review?”. Currently only 24 votes, but the great majority have voted ‘No’ (20):
http://www.against-the-grain.com/2012/07/atg-i-wonder-wednesday-green-open-access-academic-integrity-and-mimimal-peer-review/
It’s a misleading question in that it wrongly equates gold OA with minimal peer review (and also might indirectly be categorising assessment of soundness/rigour as minimal review, which it isn’t) but still worth voting.
(Acknowledgement: heard about the survey via Peter Suber, @petersuber, on Twitter)
I saw that poll and, like you, was somewhat perplexed by the conflation of these issues. So I didn’t bother to vote.
Why do you think it’s worth voting (I confess I haven’t heard of ‘Against the Grain’ before)?
I think it’s worth voting, commenting etc any place where the myth that open access journals have lower quality or minimal peer review might be spread or accepted because of lack of knowledge. I always stress in my talks and writings that peer-review quality is independent of business model and that there are good and bad examples across the spectrum. I come across all sorts of misconceptions, and from people at all stages of their careers.
Despite the shortcomings of the question in the ATG poll I think it’s still possible to cast a vote that says gold OA publishing doesn’t threaten academic integrity.
My PLoS ONE article had a minimum of 7 reviewers, and the review process took two rounds, and showed all the killer qualities of an aerial dogfight – it was among the toughest reviewing I have seen to date, and I moonlight as a freelance sciencewriter for leading scientists, routinely handling submissions and rebuttals to journals in the establishment “top tier”. I know no-one who has waltzed in at PLoS ONE.
The PLoS ONE review system has faults aplenty, but it remains the only breathing hole for truly innovative, unpopular or “unsanctioned” (but demonstrably sound) work that challenges the established hierarchy. The system is naturally asphyxiating. PLoS ONE is still the only scientific journal I know of to date that publishes objectively, rather than subjectively. The intrinsic value of that will show if PLoS ONE stays afloat long enough.
Wow! seven reviewrs! I have had three! Let us not forget. PlosOne does not publish scinetific micro or macro reviews. Thus the impact factor is solely on orginal reserach data intensive manuscripts.
With Respect to all !!! i am currently a Ph.D candidate in Molecular biology at University of Delhi, INDIA. (non-native English speaking). submitted my first manuscript (first as leading author) in PLOSONE and just received the “comments from reviewers”. i found it decent and rigorous enough to be called as a “peer reviewed ” . It is very educational to read this blog and comments from experts like you all. all i can say is …sometimes its very hard to understand so called the “peer reviewing” . one of my reviewer commented that ” why 3 pepole were needed to sample 10 soil samples (10 subsample for each composite sample) “. after design the research performing the experiments and writing the article to the best of your english skills.. if manuscript got rejected just because english level is not pleasing to the reviewer i think its a injustice somewhere… or may be not!!!! GOD Bless to all of you (great editors, reviewers, copyeditors and all). things are not same for all……. enjoy the trend!!! all you can while you can !!! (by the way my manuscript is in major revison)
Nobody seems to have mentioned the possibility that shifting the emphasis to post-publication review (though with pre-publication linguistic help where appropriate) can allow different kinds of review of the same publications for different groups of potential readers. One of the problems with the current system is that published articles, unlike published software, musical compositions, plays, motor car designs, etc. are required to be frozen permanently instead of allowing corrections (major and minor) in the light of comments, criticisms, new insights by authors, new discoveries, etc. If post-publication reviewing (organised by different groups of readers to meet their needs) and post-publication correction/amendment, etc. were commonplace we would need far fewer publications. Perhaps the use of citation counts and the like as a means of assessment, as opposed to judgement of quality of content, would be reduced, which would be a very good thing — for many reasons including the fact that some publications are very highly cited because criticising their errors can become a bandwagon. I have more on open-access here: http://tinyurl.com/CogMisc/open-access-journals.html
Interesting comments! I have about 50 papers in traditional journals-mostly society journals but not exclusively. I just sent something in to PLoS (*fingers crossed). Why? Well, most of what I do is highly interdisciplinary, and traditional journals organized along traditional academic disciplines just do not handle this well. Their spinoffs have carved the world into smaller and smaller niches, which also does not work. Second, journals also have an escalating tendency for arbitrary page limits, which is not only counter-productive in an age of increasingly complex problems, but also unnecessary when most content is delivered electronically.
My main beef is with the cost. I realize you can waive the fees, but $1350 still seems excessive. With 13 000+ articles published a year, that is close to $20 million. Hosting a website can’t cost a fraction of that (my Gmail/Google Drive account gives me something like 15 GB, or the equivalent of 8% of all PLoSOne at 10 Mb/paper, in exchange for a few adverts that I ignore), and the associate editors doing all the work are volunteers. Where does all the money go?? PLoS One is a non-profit, so someone must be making money somewhere.
I think the competition isn’t going to be mainstream journals jumping on the bandwagon, but rather a similar journal popping up and only charging a few hundred dollars a paper–or nothing and placing a few ads alongside (if scientists had a problem with ads in their journals, Nature and Science wouldn’t be the highest-ranked journals!).
PS. I’ve published one paper in PLoS One and have two in review. Fortunately, my library pays the charges.
I don’t know the specific financials of PLoS ONE, but I do know that there a are more costs in running a journal than you may think. The sheer volume of PLoS ONE will make the bandwidth needed to deliver website considerable, plus technical staff to make sure it doesn’t break. There has to be a quite large staff to keep everything moving. Markup and the production of pdfs isn’t without a cost. This isn’t a WordPress site with a bit of advertising on the side. I have no doubt that PLoS ONE more than covers its costs but then PLoS as a whole does a whole lot of other worthy things that need supporting.
It’s quite ironic that for the first years of PLoS’s existence they were criticised because they were supported by sizeable philanthropic grants. Now they get criticised for making enough money to keep them running!
If you are interested in the finances of OA you might be amused by this ‘modest proposal’ for the next concept in OA publishing.
Hi, this post has enforced my decision to submit my paper to PLoS ONE. The main reason is that I sent it to a different outlet (with a much lower IF than PLoS ONE) only to wait 13 months(!) for the editor’s INITIAL decision that it would not be of enough interest to their readers. It is fine to reject, sure, but 13 months later – despite me begging them for a decision. It is just so cruel when I could be putting efforts in to meeting targets elsewhere. Letting the reader decide if the sound science is interesting or not seems perfectly resonable to me.
I’m sorry to hear you had such a horrible experience at your initial journal and hope things go much quicker at PLOS ONE. My own experience this year was of submission on March 28th and publication of the revised manuscript on June 7th.
This year I have published four papers in PLoS ONE and found the peer review as rigorous as any other journal I have published in, if not more. I keep detailed notes including the reviews here: http://proteinsandwavefunctions.blogspot.dk/search/label/plos%20one