Can One Simplify the REF?

Posted on October 14, 2024 by Athene Donald

The REF is much in the news, with some feathers ruffled by UK Day One’s proposal to simplify the whole process, as detailed in their report Replacing the Research Excellence Framework. I am sure there are academics and administrators up and down the land who would welcome simplification, but not at any cost, metaphorical or otherwise. Ben Johnson, former Government advisor, has written enthusiastically about UK Day One’s plans but, as James Wilsdon has spelled out, simply relying on the totality of funds earned to drive the distribution of Government funding has all kinds of issues underlying it.

Indeed, Wilsdon was the lead author of the important report The Metric Tide (2015), which considered how the REF could be scrapped, something Prime Minister Gordon Brown had wanted back in 2006, and a simple ‘basket of metrics’ used in its place. This would have simplified things, although I suspect metrics beyond the single figure of already awarded grant income was always envisaged back then, but the response of the sector at the time was deeply negative. Metrics, one or many, remove any opportunity of nuance due to circumstances at any level from the individual to the institutional. The Metric Tide spelled this out in great detail. (It won’t surprise anyone that one of the objections I personally would raise is the obvious statistic about how diversity would be negatively impacted, as was made clear by Wilsdon and co-authors.)

When I chaired the REF Interdisciplinary Advisory Panel, early in the cycle for the 2021 exercise, we were asked to consider whether any of a number of metrics could possibly be helpful in assessing work that crossed boundaries. We went through the list provided one by one and unanimously concluded that no, none of them was likely to be a fit measure. And so none was used.

It is tempting to think that using someone else’s decision about what is ‘excellent’, as would be the case if funding were the sole criterion, would be a sensible choice. However, in practice it simply amounts to outsourcing a decision to other, non-calibrated bodies. Even if one stuck with UKRI funding and assumed that all successful UKRI grants were equally excellent, I fear the evidence would not support that quantification. Let me explain why, directly from my own experience.

Back in the ‘olden days’, before UKRI or even BBSRSC was a thing, I sat on a panel of one of BBSRC’s predecessors, the AFRC (the Agriculture and Food Research Council). At that time grant-holders had to write a final report saying what they had achieved with the money they had been given, and we had to assess them. ‘We’ in this case being the same people (since these were standing committees, which met at regular intervals) as had judged the grant good for funding in the first place. It was dismal to see how many grants we must have raved about three or more years previously were then graded poorly once we read the final reports. Now, of course some failures are to be expected if exciting but risky stuff is to be supported, but too often the outcomes just seemed boring, incremental or non-existent. It made for sobering reflections, as we tried to work out why we got things so wrong.

Some of this may have been hype in the original proposal, promising the moon and we were too naïve to see through this. Some of it would undoubtedly have been due to circumstances beyond their control. I can well remember a (BBSRC) grant of mine which got precisely nowhere, largely because the amount of time the central instrument was functioning properly was so limited. We – by which I of course mean the poor postdoc – did our best, and they didn’t waste their time as they tried to work out where the problems lay when the manufacturers weren’t particularly helpful. But, as far as I recall, we only managed to write one paper and that not of a very high quality. These things happen. But too often, reading the final reports it just felt as if the grant-holders had either lost interest or been buried in other tasks so as to be unable to drive the specific research programme with adequate attention. (I may say I don’t believe our assessments made any difference to anything, although obviously in principle bad final reports could have been used to blackball a particular researcher for some time.)

That is clearly a single data point, but highlights with the best will in the world panels making judgements will sometimes get it wrong. The process is inevitably flawed and to allow further money to flow based on it would just exacerbate inequalities. It really is surely better to judge outputs, and judge them with a human eye. However, beyond that rather major problem, it is worth thinking about unintended consequences if this metric replaced the REF. If grant income is what matters, then more people will be writing more grants, and will feel themselves under pressure to write more grants. More panel members will need to be found. More personnel at the funders to administer the proposals and more administrators in the universities to do the costings… and so it goes on. The costs labelled ‘REF may disappear, the bureaucracy that we all recognize now may disappear, but it will turn up elsewhere to service this different ‘simplified’ approach. That really isn’t a solution.

But there is another fundamental problem that even the ongoing REF is still wrestling with. What price excellence for the people working on grants? Just as professors will feel themselves under more pressure to write more grants, once they have been awarded one that pressure will likely get transferred to the researchers on the grant to produce results at speed to make it easier to obtain further funding. There will be more emphasis on more papers, more hours at the bench, more competition to get that Nature paper out swiftly, and less room for work-life balance, compassion and support as people try to find their way through the academic maze. The research culture element is still being argued over, but as a community we should not allow excellence to be interpreted only with regard to outputs and not take people into account. I fear a funding metric would be intensely detrimental to our lab culture, just when people are beginning to take it seriously. We do not need another generation looking just like the present generation, learning how to compete, bully or fail to bring out the best in their teams. It really is a depressing thought.

There may well be a better way of analysing research excellence, but relying on a crude single number is not likely to be it.

 

 

Posted in Science Culture, Science Funding | Tagged , , , | 1 Comment

The Path Not Taken

Posted on October 6, 2024 by Athene Donald

One of the last tasks I did as Master of Churchill College, was to partake in an afternoon’s event as part of their alumni weekend, badged as ‘Arts meet Science’. The first, and more substantial part, consisted of various pieces of music, mainly new – or at least newly configured. It had a strong emphasis on women, not least due to the involvement of the Marsyas Trio, who have an ongoing association with the College and who contributed to several of the works. The trio are not only women themselves, but are champions of music by women that doesn’t always get the attention it deserves, as their repertoire makes clear. In this particular concert they showcased a piece by Fanny Mendelssohn-Hensel, her Overture in C, which had been arranged for them by a previous Director of Music at the College, Mark Gotham. Hensel was so much more than Felix Mendelssohn’s sister, however much she was stuck in his shadow throughout their short lives, and restricted in what she could do by societal and familial pressures. (If you want to know more about her, I would recommend Anna Beer’s Sounds and Sweet Airs, telling the story of several largely forgotten but impressive women composers.) Mark Gotham, it should be said, exactly exemplifies the ‘arts meets science’ label, as he is now a member of the Cambridge Computer Science Department, having previously been a Professor of Music Theory in Germany.

It would have been possible to make the connection between this musical section and my own talk, in which I briefly discussed my recent book (Not Just for the Boys: Why we need more women in science), by stressing the issues facing women, past and present. Instead, the current Director of Music, Ewan Campbell decided to emphasise the fact that once upon a time I had played the viola. A long time ago! Music was incredibly important to me as a teenager (you can hear more of my experiences by listening to a rather old Desert Island Discs).

It was a respite from a heavy A Level load and a place where friendships formed based on what we were doing together in choir and orchestra. But it was not something I wanted to pursue as a career, despite the urgings of my teacher. I knew I was ‘in demand’ as a viola player, because they are typically in short supply; I knew as a consequence, I got to play with some wonderful musicians who did proceed to professional careers. To play the Schubert String Quintet with four individuals, all of whom made music their lives was a fantastic experience. I was not in their league. When my viola teacher wanted to persuade me to study at one of the London colleges I could immediately see it would amount to a life simply of being a peripatetic teacher, like her, and it did not appeal. Cambridge and Physics beckoned and I never regretted that.

However, the reality is that many teenagers are totally perplexed by what they ‘should’ do. Should, in the sense of a parent or teacher pushing them in a direction they may or may not be comfortable with. Or ‘should’ in the sense that they know little about career choices but someone once mentioned to them that career X is a safe, or interesting, or financially-rewarding career, and therefore they think that must be the right choice, regardless of their interests. England is unusual in forcing teenagers to make choices at an early age, when they know little about themselves, little about the world beyond school and are also very susceptible to peer pressure, which may be ill-informed if not actually ill-intentioned. So, we have a system where too many people make choices that aren’t right for them and/or are not wise.

I believe this early decision making strongly influences girls to steer clear of subjects like Physics, which means our workforce is less diverse than it should be. However, it is dismaying that often schools are unable to provide adequate career advice, due to a lack of resources. Too often A Level choices are not made wisely for a particular career path. A common problem is the belief that taking a single science A Level so as to keep one’s options broad will prove adequate to study some science at university. Usually this isn’t so. Nevertheless, as the Royal Society has been recommending for years, a ‘broad and balanced’ curriculum is really important for the 21st century, so that teenagers don’t get forced down narrow paths that then equip them poorly for the world of work later on. To make this work would require many substantial changes in the educational system.

For those for whom university may not be the right choice, navigating the complex system that provides other qualifications, including apprenticeships, can prove to be an impossibly confusing challenge. The funding and qualification landscape is currently tricky to understand and traverse. It is to be hoped that the creation of Skills England will resolve some of these issues regarding tertiary education, and that the ongoing Curriculum and Assessment Review will consider how best to prepare students for their lives. There is a long way to go.

I feel fortunate, not that I had good careers’ advice – I didn’t get any at all, back in the day – but that I wasn’t confused as to what choices to make. I always knew what I wanted to do (although I would have liked to do German A Level too, but the timetable made that impossible). Many children are less clear in their aspirations, or steered in unhelpful directions that don’t bring out the best in them. For every child there are many possibilities, but at each stage doors tend to close, narrowing options. We may not have a Labour Government that talks in terms of ‘Education, education, education’, as Tony Blair did in the run up to the 1997 election. But we do have a government that is giving skills a higher priority that has been the case in the recent past and, with its mission of breaking down barriers to opportunity, it intends to start with the earliest years of a child’s life.

Each of us are constantly faced with forks in the road, even if we’ve gone a long way down one track before even noticing the decision that was silently made at that fork. I don’t regret not becoming a peripatetic viola teacher but far too many adults will look back at decisions taken by or for them that forced them in a direction that, with hindsight, was a mistake. It is to be hoped that with renewed focus on early years, schools and adult (re)upskilling there will be fewer adults who take a mistaken fork and end up somewhere they wish they’d never got to.

Posted in Education | Tagged , , , | Comments Off on The Path Not Taken

An Anthropological Experiment in Birmingham

Posted on October 1, 2024 by Athene Donald

I’m not sure that spending my last day as Master of Churchill College at the Conservative Party Conference would have been quite what I expected, but so it was. I was in Birmingham – just as I was in Liverpool last week for the Labour Party Conference – to support the Royal Society’s work around Science 2040. This is their project taking a forward look at what the science system ‘could and should’ look like by 2040. I’m delighted to be associated with it, both as a steering committee member and leading an ongoing piece of work about how career pathways need to develop. An interim report on the project will be published in the spring of next year, but already one strand has been published regarding what science does for the economy. With growth much in the Government’s eyes and words, spelling out the important, if often hard to quantify, contribution science makes to the economy is crucial.

This piece of work has been led by Richard Jones, my friend and erstwhile Cambridge colleague, now Vice President for Regional Innovation and Civic Engagement at the University of Manchester as well as Professor of Materials Science there. He gave an eloquent introduction to this strand at the reception in Birmingham, stressing how much Birmingham itself, over the last two centuries and more, has contributed to the science and innovation base of the country and how that had significant impact for the economy. Birmingham is of course associated with the Lunar Society of Erasmus Darwin (a polymath and one of my heroes), Josiah Wedgewood, Matthew Boulton, James Watt and more.  It may have been a dining club, named after the fact that they met at full moon when it was safest to travel home late in the evening, but it was far more than that. Their friendship and inventions were a sparkling example of how science and innovation go hand in hand and can change the world. The contributions to the economy and our well-being in general are as important now as they were as in the Lunar Society’s time in the last years of the eighteenth centurey. (I’d recommend Jenny Uglow’s book about this group to you if they are unfamiliar to you.)

Attending party conferences is, as someone described it to me, an anthropological experiment. They are certainly unlike any academic conference I ever attended in many respects, if not in all. One similarity I noticed (and in this I was not alone) was that the percentage of women present was low, far lower than at the Labour Party conference. Furthermore, the dress code was very different, even if it was an unwritten rule. Men, overwhelmingly, were in blue suits with white shirts; tie choice was free, but ties themselves did not seem to be optional. In Liverpool the range of clothing for men was distinctly more varied, although I was surprised to see how many women were in dresses (the latter unlike academia, the former much more so). It was sheer chance, but just to confirm stereotypes I passed a couple of what I assume were recent graduates allowed out on the circuit, with one saying to the other ‘since you’re an old Etonian….’.

However, anthropology and more science apart, I mainly attended sessions on skills and apprentices, of which there were many (in Liverpool, the skills talks seemed doomed to be simultaneous, so I actually got to fewer of them there). Asking a question at one event about the apprenticeship levy, when no one had actually mentioned the role and vital importance of the providers as opposed to discussing the employer’s point of view, I was somewhat surprised to have Robert Halfon (one of the speakers) challenge me in response about why Cambridge University was not providing degree apprenticeships – which was not particularly relevant to my question about FE Colleges, but I supposed it meant he didn’t have to answer that. Even more surprising to me was when I walked into the room of a later session, on which he was also a panel member, and he shouted at me across the room, something along the lines of ‘Cambridge is here again; are you stalking me?’. To which my reply was ‘isn’t Cambridge allowed to be interested in apprentices?’. Perhaps my red jacket was equivalent to a red rag to a bull, but I felt he was out of order.

Halfon may have a bee in his bonnet about degree apprentices, but David Willetts (now of course a member of the House of Lords) was having none of it. In a third event on apprentices and skills, Lord Willetts made it absolutely plain he could see no logic in this particular hang-up, explicitly naming both Robert Halfon and Gillian Keegan in this context as having focussed far too much on this. He emphasised that in his view we should be ensuring the apprenticeship levy was spent on the under 25’s to get them into the workplace (Levels 3-5), and not providing Levels 6 and 7 for those already with jobs. All the evidence shows an increasing trend of firms sending those already with significant qualifications on to degree apprentices at the expense of school leavers. David Hughes, CEO of the Association of Colleges and on the same panel, wholeheartedly agreed with Willetts regarding this point.

In the Science 2040 strand of work I’m leading on careers’ pathways, these are some of the issues we’ll be exploring. I feel it is very important that the Royal Society explicitly recognizes that the elite scientific system of its Fellows often rests crucially on the shoulders of others whose qualifications are much more modest, and I was delighted that its five-year strategic plan explicitly recognizes this.

So, as of today and now I’m back in a very wet Cambridge, I am fully retired. It is a very strange sensation. I will miss the day-to-day business of a job and specifically of my colleagues at Churchill College (although I now become a Fellow Emerita). But it gives me the opportunity to explore new avenues – and perhaps write more.

Posted in Education, Science Culture | Tagged , , , , | Comments Off on An Anthropological Experiment in Birmingham

What Can I Do to Help?

Posted on September 15, 2024 by Athene Donald

Men who’ve heard me talk about my book (Not Just for the Boys: Why we need more women in science), or more generally about the issues facing women in STEM, not infrequently ask me this question: what can I do to help? I can point them towards the helpful list I first published about nine years ago, and which I frequently use in my talks as the last slide, to leave on the screen during Q+A. But perhaps I should also point them towards the book I’m currently reading, Unrooted by Erin Zimmerman, with its subtitle of Botany, Motherhood, and the Fight to Save An Old Science. The Canadian scientist is an evolutionary botanist, and a significant part of her book is dedicated to her love of that branch of biology, and her dismay that botany, in its classical form, is fading due to lack of funding and support, while molecular approaches take centre stage.

However, the part of the book I want to stress here is that summed up in the second word of the subtitle: motherhood and all the accompanying challenges for an early career researcher. Zimmerman thought, like many another young scientist, that she wanted and was destined for a career as an academic. However, her experiences of postdoc-ing while pregnant and after returning to work after the birth of her daughter (she was only entitled to four months maternity leave), demonstrated, for her, that the battle to combine the different roles became unmanageable in the face of an unsupportive boss.

That is the key point. He wasn’t overtly hostile, even particularly sexist, he simply failed to understand what she was going through, or how her experiences resulting from what he said and did and, just as importantly, by what he didn’t say or do, impacted her. Her experiences, I fear, are far from uncommon. Yes, there are misogynistic professors out there who are much more explicitly unpleasant, or even harassers and predators. But there are plenty more (and some of them women too), who simply have no imagination or sympathy for their team. All they are focussed on are the results, the papers, and being able to get the next grant. Sadly, our system of academic incentives currently makes such a focus unsurprising. The supervisors’ survival (particularly if they are still on tenure track or its equivalent) depends on these outcomes and not the wellbeing of the researcher.

Zimmerman describes this tension, this slow destruction of her ability to keep going in the face of apparent insensitivity and unawareness of what she’s going through, in miserable detail. It’s something many supervisors would be well advised to read if they want to be able to support their researchers.

“Each time you’re made to feel unprofessional for having caregiving responsibilities, each time you’re made to feel like a burden for requesting minor accommodation…it wears you down a little more. You believe that you are the problem. And when the reward at the end of those years of hard work and low pay are far from assured, it doesn’t take a PhD to figure out you might be happier and better off elsewhere, no matter how much you love the actual science and the questions you were trying to answer.”

I am sure those words will resonate with many women who have wanted to combine their love of science with love of their child(ren), but struggled.

Yet the reality is that academia should be quite flexible. Zimmerman’s complaint was in part that her boss simply made things difficult any time she wanted to deviate from what he saw as the ‘normal’ way of working. If she wanted to start early to fit in with when childcare was available, he seems to have rolled his eyes before giving grudging approval. Presenteeism should not be necessary in academic life, although some experiments (particularly with living organisms, which by and large her plants were not) may put certain demands on timings to make sure they stay alive. But otherwise, getting the job done – including reading at home in the evenings, if that’s what works – should be the only thing that matters, not when, or even where, it’s done.

I was clearly lucky in many ways when I combined motherhood and my science. It was a different age (four months maternity leave was generous then) and people – certainly in the Cavendish – hadn’t had to think about these issues before: as I was the first female lecturer there, I was obviously the first person to try to make this combination work. Maybe this made it easier as people collectively seemed to want it to go well. No one checked what hours I worked, as long as I turned up to lecture and run the practical classes at the appropriate times (and my working hours were extremely flexible, due to when childcare was available and how my husband and I shared the rest of the week). I didn’t have to go through the indignities of pumping milk in unsuitable surroundings, as Zimmerman did, since advice on how long to keep a baby on breastmilk alone was very different then, although I kept breastfeeding in part for many more months after I went back to work. I was shattered by these early months, lecturing at 9am when I thought my legs might give way I was so tired, but at no point was I faced with disapproval or even comment.

So, supervisors in general, think a little harder about what the young parents in your team may be going through and work out how to make it easier so that they can deliver what you want. Making their life a misery through inattention, disapproval or worse, will actually make the outcomes less successful for the whole team. A period of irregular working may still be significantly more productive than allowing someone’s drive to ‘wilt’, to use Zimmerman’s word.

“To keep making my way up the ladder with no additional thought given to my happiness or comfort in the workplace was getting frustrating. I felt like after more than a decade in research, I’d earned some basic consideration.”

So, if you want everyone in your team to be both successful in themselves and contribute to the success of your wider team, show a little consideration….I think that’s the next piece of advice I should proffer when I’m asked in the future ‘what can I do to help?’

Posted in Research, Science Culture, Women in Science | Tagged , , , , | Comments Off on What Can I Do to Help?

For the Last Time

Posted on September 6, 2024 by Athene Donald

For the Last Time

I have written in the past about the challenges of doing something for the first time. For early career researchers, this could be anything from giving a conference presentation to travelling to another lab to learn a new skill or joining (and speaking up at) a departmental committee. Anything unfamiliar can feel unnerving, although the more one tries things out the more one learns (even, as I found when I gave a TedX talk, that I never wanted to do it again). Indeed, the more one tries, the more one may begin to believe that you can cope with new challenges even if you make a complete pig’s breakfast that very first time. With luck, you know you can live down any associated embarrassment and come back stronger next time around.

However, as I discussed in an earlier post, retirement from my post as Master of Churchill College is almost upon me, and I am now facing the opposite challenge. It is a weird sensation that every committee in College I turn up to chair, I know it’s for the last time. Whatever decisions are made, it is for someone else to carry out. It is not a very comfortable position to be in. Of course, I could be completely destructive and try to leave a mess behind me, but that really isn’t my style. More to the point, I am left thinking, why did I not do more? Why did I not push through that change that always lurked at the back of my mind, or encourage a junior member to be more forceful in setting out their imaginative ideas? One can always look back and feel that perhaps one was too laid back, too hesitant, too nervous…..You get the picture. Seniority does not necessarily bring supreme confidence (although, of course, it may for some people).

There will be some committees which were less exciting than others, at least for me, (that doesn’t of course mean they weren’t important), that perhaps I won’t be so sorry to see the back of. I am fortunate in that retired Masters remain Fellows at Churchill, so that I will be able to see how things that have been set in train pan out in the months and years ahead. Indeed, it astonishes me that not all colleges do this; how cruel to serve for a significant number of years and then be unceremoniously excluded from the future life of the college. I will, perhaps, be particularly interested to see how the work around sustainability develops, where Churchill has made significant strides, although there’s plenty more to do.  We are very proud of the fact that, not only have we installed solar panels on many of our, fortunately flat, roofs, but that we invested in training our maintenance staff, so that they have been able to do the installations themselves.

However, what this amounts to is that overall, doing things for the last time is definitely bittersweet. It is right and proper to move on: the historic habit of Masters serving for life is definitely something that should never be restored. One gets stale, apart from anything else, unable to see new solutions to old problems which a newcomer may spot instantly with their fresh eyes. I’m looking forward – even if with significant trepidation – to whatever comes next. I hope it will raise new challenges to stimulate (and the signs are starting to be positive on this front), new things to do for the first time, not the last. I hope I will still find the issues, quirky or deeply serious, which provoke me to continue with writing this blog, but I look forward to new horizons, situations, even committees to keep my brain agile and my service to the community non-negligible. Time will tell.

 

Posted in Careers | Tagged , , | 1 Comment