Systematic Errors of Judgement

To tie in with International Women’s Day, last month Nature ran a series of articles about the issues still facing women in science and also a podcast with Uta Frith and myself debating some of the issues. The interviewer, Charlotte Stoddart asked me, amongst other things, why I felt unconscious bias was so important that I frequently wrote about it.  My answer was, and is, that it is important for the very reason that it is unconscious, unseen unless taken out of the closet and inspected and so liable to be invidious and pernicious. It is, if you like, a systematic error distorting the very way we judge and are judged – in general to women’s detriment.

Much is made of the practical difficulties of combining a career and a family for women; this is often proposed as the major reason for women leaving science, but I think this is too simplistic. Indeed, academia is probably an easier place – because of the relative freedom we enjoy to work at times that work for us and not just clocking in and out – than some other professions for flexibility around caring responsibilities, for both men and women.  Perpetuating the view that the combination is well-nigh impossible (without exploring, as Ottoline Leyser’s excellent book did, all the different ways women have found actually to manage to do it), is probably only discouraging and disempowering in itself.

But the evidence – gleaned from study after study in the social sciences and related literature, which many Deans of Science or Heads of Science Departments are probably not reading – seems to point to the range of subtle ways in which we, collectively, reach conclusions on others which, all unknowingly, are influenced by gender and stereotypes.  At the bottom of this post I list some of the papers I have come across in the past year or so, not all of which I have comprehensively studied and which I am unqualified to assess in terms of their (social sciences) methodology. I do this for the interested reader in the hopes that this may facilitate their distribution to decision-makers in their institution (some may be behind paywalls depending on institutional subscriptions, so I just link to the abstracts).  In this present post I would like to pick out just one which, like the much-discussed PNAS paper by Moss-Racussin et al, identify some disturbing aspects of the way we all – these failings affect women as well as men – reach conclusions about the work of others and then act them out (I’ll use another post to discuss a very recent paper by Wang et al on an analysis of choices boys and girls make about careers).

Unconscious bias can only be dealt with by making it conscious, by ensuring an instantaneous assessment is backed up by evidence and not just by one’s unthinking gut. Now that overt discrimination is relatively rare and explicit sexism of the sort that says ‘women can’t do this’ rarely annunciated, we need to move on to a situation where the playing field is genuinely level and not just free of gross peaks and troughs. It behoves us all, particularly those in decision-making roles but also those at the receiving end, to consider the myriad and disparate ways in which subtle cues can trigger different reactions according to the gender of the individual under consideration.  This isn’t simply a case of differing ways of reacting to actions, as in the conjugation ‘he is assertive, she is aggressive’ and equivalent pairings. It is also a case of reading between the lines of letters of reference (as I’ve talked about before here and here) or metrics (see this article) and citations (here).

Scientists may like to form judgements based on quantification, but what is emerging from the literature is that our collective reactions invariably seem to imply a systematic disadvantage to women by the use of such metrics, even though it looks like something as gender-neutral as citations may be being studied. The most recent study I came across, entitled The Matilda Effect in Science Communication: An Experiment on Gender Bias in Publication Quality Perceptions and Collaboration Interest  looked at the responses of ‘243 young communication scholars’ when asked to rate some (carefully manipulated) conference abstracts. (The Matilda effect was a phrase coined by Margaret Rossiter 20 years ago to describe the systematic underrecognition of women in science.) The abstracts’ topics and authors were varied to see how the readers reacted and to test a series of hypotheses relying on ‘role congruity’. This theory says that a group (or in this case, an abstract) will be positively evaluated when its characteristics are recognized as aligning with that group’s typical social roles. So a paper written by women about a subject ‘appropriate’ to their gender, such as the effect of media on children (remember this was a project involving science communicators), will be more highly rated than one written by women on an ‘inappropriate’ topic such as political communication.

Their hypotheses were largely borne out; on average the papers written by ‘men’ were perceived as of higher quality than those written by ‘women’, and even more so if stereotypically male topics were being written about. The respondents were also more likely to want to collaborate with the males on stereotypically ‘male’ topics and with the women on those topics associated with women. These trends were the same irrespective of the respondents own gender. The differences in evaluations were not large, but as earlier studies have shown, small effects multiply up over time; this is true of salaries and it is true of less tangible attributes such as recognition or collaboration opportunities.

Somehow we – again I’d stress this seems to apply to both men and women – react differently when we see a name on a piece of paper (let alone when we meet the individual) and are less likely to cite a woman or want to collaborate with them. This reaction is apparently so in-built and rapid that it can happen without even noticing that one has taken in the gender of, say, a paper’s author. I have always believed I don’t look at the names at the top of papers – including those for job applications – and so am unbiased. This latest paper suggests that, whether or not I have taken in a name sufficiently to be able to recall it, I may still be being influenced, so I will have to work harder on this.

I have frequently heard calls for reviewing (eg for grant applications or jobs) to be done blind, i.e. with the name removed. I am not convinced, certainly within a country as small as the UK, that this would be meaningful. You cannot remove details of the field being worked in, the papers cited, or the list of publications attached, so how would an anonymous refereeing process actually work? Surely one should keep names in and remind everyone, constantly, to be on their guard for their own hidden biases? But doing the latter is hard, and cannot be dealt with simply superficially.

To illustrate this let me finish with two linked anecdotes (I know, this isn’t data). The first is from a woman who enquired about how unconscious bias was dealt with at a research council panel. She was told ‘any funny business is stamped on pretty quickly‘. Now, what did the official mean by this? That illegal discrimination was spotted and dealt with, or that anyone who has lurking unconscious bias is hauled over the coals? I think it’s easy to see which was intended. But by way of a specific example of the dangers of bias, of the sort that could turn up at a research council panel, let me cite statements from two recent references I saw (edited for the sake of anonymity) for a male and female applicant of roughly equal standing – the woman happened to be a year further on in her career but had taken time out to have a child – both with stellar credentials.

Woman A

‘ a consistent output of more than a dozen papers per year, despite a period of maternity leave and currently working less than full time; more than £2M of current research funding held as PI….however she is still at a relatively early stage of her career and this makes me uncomfortable about recommending her….’

Man B

‘I should comment on the fact that all but 3 of B’s recent publications do not include Y [his mentor, still in the same department] as a co-author. However for about half of these B appears to be the senior author, and presumably the intellectual driving force behind the work….my overall view is that…he is highly deserving…’

I expect you can see why these two references, in juxtaposition (and with A and B in competition) made me so angry. OK, they weren’t written by the same referee, but in one case the woman is damned with being too early on for serious consideration despite overwhelming evidence for her independence – through grants and papers – explicitly being presented. On the other hand, when it comes to the man – whose independence is being queried with the referee only using weak words like appears and presumably regarding what B has achieved – there seems no doubt in the referee’s mind that B is ‘ready’.

The panel I was on read these comments and dealt with them appropriately in my view, but not all panels may be so sensitive to seeing this manifestation of the Matilda Effect in practice in letters of reference. Is this a form of ‘funny business’ that needs to be stamped on? I think so, but there is no guarantee that it will be. This is why unconscious bias has to be teased out and discussed at every opportunity. Please distribute the (links to the) papers below as widely as possible, and add more to the reading list. It would be nice to think progress could be made.

Reading List (in reverse chronological order; this is bound to be very incomplete, as it relies on papers that have happened to cross my path rather than a systematic search, and only covers the last year or so; additions to this list would be welcome).

Not Lack of Ability but More Choice: Individual and Gender Differences in Choice of Careers in Science, Technology, Engineering, and Mathematics by M-T Wang, JS Eccles and S Kenny published online March 18, 2013, doi: 10.1177/0956797612458937 in Psychological Science
The Matilda Effect in Science Communication: An Experiment on Gender Bias in Publication Quality Perceptions and Collaboration Interest by S Knobloch-Westerwick, C Glynn and M Huge published online February 6 2013 in Science Communication
Is publication rate an equal opportunity metric? EZ Cameron, ME Gray and AM White Trends in Ecology and Evolution 28 (2013) 7-8
Science faculty’s subtle gender biases favour male students CA Moss-Racusin, JF Dovidio, VL Brescoll, MJ Graham and J Handelsman PNAS 109 (2012) 16474–16479
‘I wouldn’t say it’s sexism, except that … It’s all these little subtle things’: Healthcare scientists’ accounts of gender in healthcare science laboratories V Bevan and M Learmonth Social Studies of Science 43 (2013) 136-158
Penalties and Premiums: The Impact of Gender, Marriage, and Parenthood, on Faculty Salaries in SEM and non-SEM Fields  K Kelly and L Grant Social Studies of Science 42 (2012) 869-896
The academic jungle: ecosystem modelling reveals why women are driven out of research KR O’Brien and KP Hapgood Oikos I21 (2012) 999–1004


This entry was posted in Science Culture, Women in Science and tagged , , , . Bookmark the permalink.

15 Responses to Systematic Errors of Judgement

  1. Pingback: Athene Donald on unconscious bias | Feminist Philosophers

  2. Geologist says:

    Thank you for these!! It is difficult for me to keep up on the social sciences when I have so much else to do within my own fields.

    Although I doubt that I can get any of my administration to read or take ANY of these papers (or this topic in general) seriously, that doesn’t mean I will stop trying from time to time.

  3. Erin says:

    This study about “The Impact of Gender on the Review of the Curricula Vitae of Job Applicants and Tenure Candidates: A National Empirical Study” may be pertinent; it proves a gender bias in both male and female participants towards male-identified job applicants.

  4. MZ says:

    Thanks so much for this; I am doing a presentation on science and gender at a big university soon and desperately needed to update my references. In my experience people are oddly unwilling to admit that bias can be happening if they can find the least bit of justification for ignoring a study, like, for example, dismissing it because it’s older than a couple of years.

  5. Pingback: gender bias: the Matilda Effect | My Life in Order

  6. Pingback: Around the Web: Cool linky stuff for science undergrads (3) – Confessions of a Science Librarian

  7. Great post. It is interesting that people become very uncomfortable with the idea that their decisions might be affected by gender bias. It may be useful to consider decision-making in a Bayesian framework. When we make decisions, we combine two things: first, a likelihood (the sensory evidence: perhaps a candidate’s resume or her job talk). Second: a prior (some estimate of how probable it is that the person is a good candidate). In many decisions, even simple perceptual ones that don’t involve gender, we seem to have more conscious access to the likelihood than the prior. This means that we can’t tell the degree to which we even have a bias or are applying it. It makes the problem a tough one to fix!! Anyway, let’s make peace with priors and move on from there.

  8. BB says:

    Thanks for this post. I second the fact that up to date references are useful.

    Of course not only will the timeliness be questioned but it is also safe to ignore anything not published on academics from your exact subject, exact country…in fact if it isn’t about your specific department then you may as well ignore it….

    It is a constant wonder to me that scientists fail to apply the basic rigour they use unfailingly in their subject area to any other realm of knowledge. If there is a weight of evidence in a field that supports a particular hypothesis then you actually need something a little more than ‘well I feel it might not apply here’ to over turn it!

  9. Pingback: Friday links: Canada’s greatest know it all (no, not Jeremy…), the research productivity of moms, HS teachers on EO Wilson, and more | Dynamic Ecology

  10. Pingback: Morsels for the mind – 12/4/2013 | Six Incredible Things Before Breakfast

  11. Pingback: Around the Web: Yet more librarian angst, The business of literature and more – Confessions of a Science Librarian

  12. Catherine says:

    Came across this paper recently, it has some depressing quotes in from scientists:

    “Gender Segregation in Elite Academic Science”,

  13. Nonsequitur says:

    Wow – with such sexism rampant, of course young women aren’t growing up to be scientists… how are things ever going to improve? Disney’s range of t-shirts – girls ones for those who ‘need a hero’ and boys ones for those who want to ‘be a hero’

  14. Pingback: Sticking with initials | Why Science Is Sexist

  15. Pingback: Systematic Errors of Judgement | wuawn