{"id":6113,"date":"2021-01-17T10:34:31","date_gmt":"2021-01-17T09:34:31","guid":{"rendered":"http:\/\/occamstypewriter.org\/athenedonald\/?p=6113"},"modified":"2021-01-17T10:34:31","modified_gmt":"2021-01-17T09:34:31","slug":"are-journal-editors-biased","status":"publish","type":"post","link":"https:\/\/occamstypewriter.org\/athenedonald\/2021\/01\/17\/are-journal-editors-biased\/","title":{"rendered":"Are Journal Editors Biased?"},"content":{"rendered":"

Last week a paper<\/a> by Squazzoni et al appeared, which had analysed submissions to 145 scholarly journals to look for gender bias in acceptances and across the whole editorial process. They claimed not to find it. When I saw the headline I was puzzled. A careful analysis<\/a> of their own publications by the Royal Society of Chemistry in 2019 had found significant evidence of bias across their editorial process, as I wrote about previously<\/a>. Why the difference? Was this latest study different because it looked across multiple journals and disciplines, not just chemistry? Or was it different because the data they had access to provided less insight? My conclusion is the latter interpretation is correct. This recent study made some assumptions about where the problems might lie and focussed fairly narrowly there.<\/p>\n

Squazzoni et al claim in their conclusion that<\/p>\n

‘peer review and editorial processes do not penalise manuscripts by women’<\/p><\/blockquote>\n

and that, in some fields,<\/p>\n

‘manuscripts written by women as solo authors or co-authored by women were treated even more favourably by referees and editors’.<\/p><\/blockquote>\n

This conclusion seems solely to be based on the fact that the acceptance rates were, in some cases, slightly higher for women than for men. Unfortunately, they excluded some key steps and issues in their study, presumably in part because they didn\u2019t have access to them and in part because they didn\u2019t fit the hypotheses they were aiming to test:<\/p>\n

\u2018We concentrated on three possible sources of bias, i.e., the editorial selection of referees, referee recommendations, and editorial decisions, and examined all their possible relationships while controlling for important con\u00adfounding factors such as journals\u2019 field of research, impact factor, and single\u00ad versus double\u00adblind peer review.\u2019<\/p><\/blockquote>\n

So, they concentrated on some topics to the exclusion of others. The first step in the review process \u2013 did the editor even send the paper out for review, a huge gatekeeping event \u2013 was not explored at all. Yet, certainly for the highest impact factor journals such as Nature, this step is absolutely crucial. Many papers are simply rejected out of hand \u2013 but which ones and by whom? To conclude that editors treated women more favourably, at least in some cases, when this first hurdle is totally ignored seems somewhat na\u00efve. Particularly so, when there is already evidence that this step has a detrimental effect. The RSC report states<\/p>\n

\u2018we found that initial submissions from female corresponding and first authors are more likely to be rejected without peer review….\u2019.<\/p><\/blockquote>\n

By excluding the challenge of this very first step, the recent study has to be regarded of dubious use as evidence that editors do not<\/u> exhibit bias.<\/p>\n

Another issue that neither they nor, indeed, the RSC considered was the length of the review process or how many revisions were requested before acceptance (or indeed rejection). Again, one can see how easily subtle bias can creep in here. For instance, if a male corresponding (or first) author is more likely to be given the benefit of the doubt on first submission when a similar woman is not but instead revisions demanded, then there will be a consequent delay for the woman. Tying in with this concern are the results of a study<\/a> of NIH grant holders from a couple of years ago which looked at how men and women react to feedback about grant applications; it is hard to see why the conclusions should differ when it comes to referees\u2019 comments. Their conclusion was<\/p>\n

\u2018that gender bias in grant reviews (i.e., greater likelihood of highlighting inadequacy in reviews of women\u2019s grants), along with gender differences in responsiveness to feedback, may contribute to women\u2019s underrepresentation in academic medicine\u2019.<\/p><\/blockquote>\n

It seems to me that referees\u2019 comments about a manuscript are likely to have the same dampening effect on a woman\u2019s motivation, confidence and hence career progression as comments about a grant application as this study demonstrated.<\/p>\n

As yet another study<\/a> showed, inappropriate referee comments are common, but are likely to have a much more damaging impact on minorities. These authors specifically identified that<\/p>\n

\u2018future studies should test if receiving an unprofessional peer review leads to different acceptance outcomes depending on gender and\/or race\/ethnicity.\u2019<\/p><\/blockquote>\n

but, as far as I know this has not been done anywhere. \u00a0If an editor reads a negative review \u2013 however unwarranted \u2013 arising from gender bias, then they are more likely to require women to resubmit their manuscript than a comparable man. Receiving this more negative feedback because of bias in a referee, or being expected to go through multiple revisions because of how the editor reacts, will inevitably be disheartening, with the likely consequence that the discouragement will feed through into subsequent behaviour, even if ultimately the paper is accepted. Just looking at the crude statistics of acceptance is insufficient \u2013 as this most recent paper does \u2013 and may mask one of the subtle reasons why women are less likely to submit manuscripts, particularly to what are perceived as the \u2018top\u2019 journals.<\/p>\n

Finally, it is inadequate to imagine that by seeking out more women referees the problem will be solved. As Melinda Duer and I wrote<\/a> in the THE in early 2019,<\/p>\n

\u201cIt is not just men who are biased against women; so too are women, as a 2012 study of job applications published in PNAS<\/a> showed. So increasing the number of women in the reviewer pool, while giving more women useful experience, is unlikely to affect the number of female-authored papers accepted. Nor, correspondingly, is having more women on the editorial team of the journals.\u201d<\/p><\/blockquote>\n

Nevertheless, this Squazzoni\u2019s paper is enthusiastic about expanding the number of female referees and editorial board members (which may not be the \u2018best\u2019 use of any particular woman\u2019s time), under the belief that then women will be more likely to submit manuscripts and participate more generally. To quote their abstract<\/p>\n

\u2018However, increasing gender diversity in editorial teams and referee pools could help journals inform potential authors about their attention to these factors and so stimulate participation by women.\u2019<\/p><\/blockquote>\n

This looks to me like a fig leaf, and smacks of fixing the women not the system. If women suspect there is bias in the process they may be right (as the RSC report shows).<\/p>\n

For all these different reasons, I am not convinced by this study. It worries me that the paper will provide grist to those who want to believe the conclusions about a lack of bias in the editorial process, implicitly blaming women who are simply not able to ‘man up’ sufficiently, without thinking deeply about what the evidence actually demonstrates.<\/p>\n","protected":false},"excerpt":{"rendered":"

Last week a paper by Squazzoni et al appeared, which had analysed submissions to 145 scholarly journals to look for gender bias in acceptances and across the whole editorial process. They claimed not to find it. When I saw the … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":18,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10,5,27],"tags":[1442,582,1285,1443],"class_list":["post-6113","post","type-post","status-publish","format-standard","hentry","category-research","category-science-culture","category-women-in-science","tag-publications","tag-referees","tag-royal-society-of-chemistry","tag-squazzoni"],"_links":{"self":[{"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/posts\/6113","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/comments?post=6113"}],"version-history":[{"count":0,"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/posts\/6113\/revisions"}],"wp:attachment":[{"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/media?parent=6113"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/categories?post=6113"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/occamstypewriter.org\/athenedonald\/wp-json\/wp\/v2\/tags?post=6113"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}