Depersonalising the REF

Successive rounds of the RAE and its successor REF have always caused high levels of stress and anxiety. The associated workload is very substantial for institutions (and many individuals); the stakes are high in terms of both reputation and funding. Each round the rules have changed somewhat. Nevertheless, collectively academia has understandably attempted to play the system to their advantage and any new hurdle just provides an opportunity for a new way of side-stepping it.

The Stern Review (Building on Success and Learning from Experience) sought to take a long hard look at how the REF had played out and to come up with some revisions, acceptable to both the community and HEFCE, which might, amongst other improvements, reduce the scope for game-playing and mitigate the heavy administrative burden. Many of the suggestions were greeted warmly. However, one particular area that caused disquiet, particularly amongst early career researchers (ECRs) was the concept of ‘non-portability’.  I was never as persuaded as some that this was such a bad idea, although my views on this seemed somewhat controversial.

However, recently I have been a member of a working group at the Royal Society which has been preparing the Royal Society’s response to HEFCE’s proposals for implementing the Stern Review and which has tried to take a long hard work at what might provide a consistent framework for dealing with the people who populate the REF. Consistency is important and the Stern Review was did not achieve it: individuals were to be removed from the stigma of whether or not they were to be submitted by virtue of ‘everyone’ being submitted, yet still outputs were to be associated with them even if the apposite  number could be varied. So one key question that has been doing the rounds is whether the lowest number of outputs to be associated with an individual was to be zero or one. In my view, as soon as this number is anything greater than zero there is the danger of stigma being reintroduced. For those researchers with a non-standard career trajectory all the issues over ‘special circumstances’ that we saw in the last REF, and which were highly burdensome, are reintroduced for instance. And the arguments about portability for those who – just as ECRs were so anxious about – move institution become manifest. The ECRs feared that was no new institution would hire them if they couldn’t bring their stellar-but-already-written outputs with them.  (It is worth noting that this group were represented on the RS working group, and were engaged in developing the response).

The Royal Society, in its response to the proposed REF reforms on March 17^th, has tried to reintroduce consistency by removing the identifiable individual completely. Instead, the number of outputs will be determined by a new Unit of Assessment (UoA)-specific volume-measure based on an estimated, averaged over time, headcount.. In this way no one has to identify whether Dr X is or isn’t ‘included’. Following on from that, since outputs are not to be traced back to the individual, the issue of portability simply evaporates. Instead, outputs must be demonstrably associated – using a textbox to explain how – with the institution submitting the output. This means any single output can be associated with more than one institution, removing many of the anxieties for those who write monographs over several years (and potentially several universities) or for collaborative researchers working in large teams.

Without the need to associate outputs with individuals, many of the perverse incentives apparent in previous rounds would appear to disappear. (That is of course not to say that ultimately other perverse but unforeseen incentives may arise in due course): for instance the hiring cycle should have fewer peaks and people won’t be hired on meaningless low level and short term contracts around a census date. To make this work as envisaged, it is important that the portfolio of outputs is seen within the context of the research strategy set out in the environment statement. This means that assessment panels should do more than simply evaluate the quality of each output. They will need to satisfy themselves that the submissions from the UoA and institution support the stated overall strategy for supporting a breadth of excellent research. Evidence might include collaborations with other UoAs and external partners as well as other activities such as the production of new research instruments and the support of research database tools. Thus the complete set of submitted outputs will need to demonstrate that they cover all the important aspects of research activity the environment statement claims.  Providing an environment where the best research can thrive and then highlighting the excellent research that is forthcoming through an appropriate collection of outputs will be the name of this particular game.

If such an approach is accepted – and because this is a more radical approach than HEFCE and others may have envisaged we recognize that acceptance of such ‘depersonalisation’ cannot be taken for granted – then it should provoke a change in attitudes within our universities to think more about the collective endeavour and less about the grand ideas of an individual Professor Bigwig. Indeed the Professor Bigwig themselves may need to think more about their team as a team and less about them as many pairs of hands delivering papers in (dare I say it) high impact journals.

This is not just a mantra for outputs. Impact case studies are already being proposed at institutional level: I believe (wearing my hat as Chair of HEFCE’s Interdisciplinary Advisory Panel, a panel which is still in gestation) that this idea as well as potentially outputs at institutional level may provide a very  direct means of recognizing research that does not fall neatly into individual UoA silos.

I believe that such a radical rethink of how the REF operates offers a new opportunity to measure things we value – the research we do, the environment in which we do it and the outcomes derived from it to the benefit of wider society – without reducing individuals to the level of fretting over whether they personally have 4 4* publications or, if not, whether their job at risk.

Also of interest is fellow working group member and Chair of the Royal Society’s Science Policy Advisory Group Ottoline Leyser’s article in Research Fortnight.