I didn’t watch the second Trump-Clinton debate, but it is clear from all I’ve read that one of the former’s tactics to attempt to disconcert Clinton was to try to intimidate her physically – by sheer bulk and position on the stage. It doesn’t seem to have been a very successful strategy in this case, but it seems all of a piece with the man. Intimidation is of course known in other non-political sectors, indeed just about everywhere. Science is not immune to it, although it isn’t likely to be anything quite as explicitly tangible as simply physical size.
I think of this as the ‘mine’s bigger’ school of science, and it could cover anything from grant income to h index. Unfortunately, these easy metrics are not just used by one individual against another – although such implicit one-upmanship may turn up in pub ‘banter’ as well as over the lab bench – but also in recruitment, promotion and grant panels. I’ve forcibly been reminded of this in two recent committee meetings, both concerning – unsurprisingly – the issues around gender in science. Ultimately both discussions revolved around how we measure success and excellence. One was in the UK (the BEIS Diversity Steering Group), the other covering European science (the ERC’s Gender Balance Working Group). Are we, as a sector, too hung up not only on metrics of dubious worth, but also on an out-of-date set of criteria overall?
After the BEIS meeting I went back and read what I wrote when my University released its Meaning of Success book in 2014. I believe that what I wrote then still holds true. That success should cover much more than mere figures and some of the interviewees included ‘building teams, seeing their students thrive and progress, working with people who sparked them off intellectually and seizing opportunities to try out new things and make new discoveries.’ How do you measure such things? And if they aren’t measurable in a quantitative manner, how can an appropriate narrative or qualitative presentation be evaluated?
Equally importantly, if one is using narrative as a determination, can this be made objective? The ERC (both the Gender Balance Working Group and the full Scientific Council) were shown a video about unconscious bias which very concretely demonstrates the dangers of using ‘gut feelings’ and subjective measures. Recognizing that there is a danger of falling between the Scylla of slavish metrics – where bigger means better but it is at least an objective measure – and the Charybdis of personal likes and dislikes – where there may be nothing objective concerned at all – what should panels do?
Firstly, it is clear that there are some absolute no-noes. A discussion of caring responsibilities, for instance, should not be admissible (see the video I give above for a concrete example of how such ‘information’ can sneak into discussions). It is true that there is no reason why men should not be asked how they are to manage with a newborn or elderly parent just as much as women, but the reality is even if they were I doubt their answers would be treated similarly. Secondly if the size of grant income cannot be used as a ‘pure’ number, the question is how can it be handled. In interviews it is probably possible to interrogate what it means, to discover whether a mega-grant income simply means a huge number of slaves in the lab in whom the PI takes no personal interest or alternatively a vibrant community who are individually supported. Reading a standard CV it is hard to see how an equivalent dissection of what underlies the metric of income could be carried out, but cover letters could be required specifically to include support and training of group members as a topic to be discussed.
The metric of h index or citation profile has been much more widely critiqued and is probably already being treated with caution by at least some panels. It is of course a number which is not totally irrelevant. Without believing that publication in a high impact journal like Nature necessarily means the peak of academe, I do believe if all publications are in the Journal of Neverbeenheardof one might question the quality of outputs. But to me that is the point. To decide, based on one or two papers in high impact factor journals that the output is first class begs two questions: firstly, whether a high impact factor journal necessarily always publishes truly excellent science; and secondly, that in some disciplines results may be much better written up in a series of papers in good journals rather than in one single stellar paper (for some fields they are not even well-covered by one of the nominally Superstar journals). The difference between disciplines, even sub-disciplines, in publishing and citation behaviour is also why the h index has to be used with an enormous pinch of salt as a comparator if it is to be used at all.
Now many people who have sat on panels of different sorts may be throwing their hands up and saying of course their panel doesn’t use slavish metrics in decision-making. At interview stage that may indeed be correct. But what about what happens before the panel meets the applicants, at the long- and short-listing stages? Faced with a large pile of papers to whittle down to some manageable number, it is all too easy to resort to the shorthand of metrics to choose the ‘top ten’ of CVs to consider in more detail. I was once told by a Faculty Dean how he had instructed all the departments in his remit, if uncertain whether to include a woman in a longlist or not, to keep them in for further consideration. After this edict he discovered how suddenly the number of women appointed across the Faculty shot up. Nothing about positive discrimination here, just thinking a little more carefully about CVs that might not have satisfied the ‘mine’s bigger’ school and been eliminated without proper consideration before the final stages. This anecdote demonstrates how small actions can have a big impact.
Trying to tease out the reality behind crude numbers is important. Expanding the range of what elements are considered at each stage of decision-making will ensure a more diverse range of candidates make it through. Identifying some more qualitative skills that are used as criteria and which are expected to be touched on in a CV or covering letter will facilitate the process; attitude towards training research students, general contribution to departmental citizenry, outreach in schools and more general science communication could all be explicitly asked for – add your own favourite additional skill to this list. At the very least asking for this information indicates that these are activities that are valued.
We need to do better to ensure we genuinely appoint, promote and retain the best, not just those who can jump through hoops and sell themselves hard. As in business, I believe diverse teams – at whatever level – are likely to be more successful. The issues raised in The Meaning of Success are as important today as two years ago. Unfortunately I suspect these ideas have made little inroad into thinking across the sector, despite our aspirations at the time of publication.
