Most people seem to think the Stern Review of the REF (Building on Success and Learning from Experience), published today, has done a fine job, with (if my Twitter stream is to be believed) the exception of the issue of the portability of outputs. The interpretation being put on it is that this will somehow disadvantage early career researchers (ECRs). I am rather baffled by this sentiment, possibly because I remember a world before the very first Research Assessment Exercise. This was a world in which there wasn’t a hiring frenzy in the lead-up to some artificially constructed deadline. The academic world back then did not possess the equivalent of a football transfer window. People did not have to work out the pros and cons of throwing their hats into the ring only within this window to try to move to a permanent position, potentially thereby giving up a thriving postdoctoral experience at an inopportune moment. Since the opportunity for such a move did not arise only once every five years or so, they could choose the right time to suit their own trajectory. (In fact, in those far off days, there were practically no permanent jobs on offer at all, but that’s a different tale.)
So, for someone such as myself with a long memory, the idea that work done in one place which can’t be ported to a somewhere else (which, by the by, had done nothing to support the work in question) would somehow disadvantage an ECR seems somewhat baffling. The fear seems to me to be tantamount to saying that ECRs have no confidence that, having done one superb piece of work they will ever be able to do another at their new place of work. Why? Work done prior to a job offer will surely have fed into the fact the job offer was made, but why should the new institution get the credit for it? And why should the move hinder them continuing to prove what excellent scientists they are, whether this is counted this time around or in some future exercise?
It has also been suggested that hires of ECRs won’t be made if prior work can’t be credited to it, since hiring a new ECR may do little to burnish their new institution’s reputation. I don’t understand this either since, even if they are appointed too soon before any deadline to make much of a contribution in their new role, if institutions only need to submit an average of two outputs per person then by extrapolation some can submit none. People can indeed be regarded as research active yet have no outputs submitted and there won’t be an institutional penalty, at least in a department of more than a handful of researchers. Instead, just as individuals can throw their hat into the permanent job-market-stakes at a time that works for them, so can institutions hire at times that suit them and their overall turnover of staff at all levels. Instead of having a wave of hiring and then in essence a job freeze for an extended period, they will be able to see a better spaced out supply of new blood.
Many of the people expressing concern will not have first-hand experience of the sort of game-playing that some institutions have engaged with in the past, games that satisfied the letter of the eligibility criteria but most certainly not the spirit. When I sat on the RAE2008 Physics Panel there were some egregious examples of institutions that hired a number of eminent overseas scholars for a few weeks (literally) around the census date so that their publications could be ‘claimed’, regardless of where the work was actually carried out or what contribution these scholars made (if any) to the wider good of the institution. The rules made such staff inclusions admissible. However panels did have the option (and in some instances did so) of marking down the institution under the environment heading, since such transient members of staff really could not be credited with much input into the departmental ethos. I am much happier with a system in which credit is given to the organisation where the work is carried out.
Furthermore, there are other aspects of game playing we can hope that departments will cease to attempt to play. One of the most important of these will be a removal of the stigma associated with who is not included, since everyone whose role incorporates research will have to be submitted. This should lead to a much more inclusive and less divisive environment for all. I believe this will be particularly the case for those whose personal circumstances in REF2014 had to be declared and investigated, however sensitively. Raising one’s hand and saying something along the lines of ‘I’ve been off sick with depression for two of the last five years and working part-time for the rest so please can I have my necessary outputs reduced’ cannot have been comfortable for the individuals concerned. (Here I would add in the caveat that I think there should be a distinction made between those who are ‘research active’ and those whose job description includes research, a distinction that I don’t think the Stern Review actually makes. If this distinction is not made I believe there will still be scope to exclude individuals because a head of department chooses to declare they are not ‘research active’ which would again be counter to the spirit of the proposals and lead to stigmatisation.)
Of course, the devil will be in the detail, as it always is. Precisely what the average number of outputs stipulated has to be to provide the best measure of a department’s strength will be one such thorny detail. How to arrive at the appropriate head count of who is in the final figures may also be a challenge: how do you count individuals who retire during the census period, for instance? Or is it just the FTE head count on an arbitrary date that matters? However, I for one welcome the removal of portability as likely to lead to fewer abuses than the model it replaces.
Dear Athene,
I am an Early Career Researcher who has moved institution within the past year. Though I can see that it is desirable to try to minimise the ‘gaming’ of the system associated with porting publications, I have two practical concerns about this review:
1) In a few years my research will be assessed by a probationary panel to determine whether or not my position will be made permanent. This assessment is linked (implicitly, but probably inevitably) to the contribution that I will make to the next REF. Though I am optimistic that my best work is ahead of me, I have now gone from having several papers that would have been eligible for the next REF to having none, due to the timing of my move in institution. This rather increases the pressure I will be under over the coming years to meet competitive probation requirements.
2) The idea that the REF credit for my past research remains with the group/lab where I did it rather than with me seems counter-intuitive. It would seem to allow institutions with people with already well established careers to benefit from the work of temporary staff, while a recently hired researcher essentially has to start from scratch in building up an assessable track record. I can imagine this introducing unfortunate strategising in the decision of when to submit important research work on the part of early career researchers trying to make themselves competitive at job interview (i.e. delaying submitting/revising manuscripts).
It also seems that there is a conflict between the recommendation that “All active research staff should be returned in the REF” and the statement that a unit would be able to submit “more than two outputs for some individuals and less (a prescribed minimum, potentially none) for others”. Wouldn’t that essentially mean that some research staff would not have to be returned? That would surely still leave a stigma attached to them. Can you shed any light on this?
Many thanks for your blog (I’m a regular reader).
To your last point, since funding is linked to volume (or at least it has been so far) as well as score profile, it is in the interests of a department to enter the maximum number of researchers they can. And, if everyone needs to be entered, as is being recommended, I don’t see how this would lead to research staff not being entered. This I think also would reduce the liklihood of being people being shunted onto teaching only contracts, another argument I have heard voiced.
Of course departments can up probationary expectations, but that could be done regardless of the REF. They can always set the targets they want. The central thrust of the Stern review is to put less emphasis on the individual when it comes to the submission, precisely to avoid some of the issues that are being highlighted. As I say the devil is in the detail, but it seems to me one way of mitigating many of the fears that are circulating will be to put increased weight on support for ECRs and policies about mentoring, career progression etc in the scoring of the Environment element.
As for where credit should sit, I accept that I wrote my post as a scientist and I can see that for disciplines where monographs are the norm my arguments may be less relevant. This may mean that different panels might wish to view ‘ownership’ differently, but in a science lab where infrastructure is so important, it does not seem unreasonable to me that the credit should remain physically where the work was done.
Regarding the last point of Anonymous ECR, the Campaign for the Public University has an article on this. It makes an important observation which I haven’t seen anywhere else in comments on the Stern report: the Stern proposals will lead to more gaming rather than less. The most relevant paragraph is:
“The negative consequences of selectivity for staff morale will also be accentuated because selectivity involves more intense performance management and internal processes of ‘picking winners’. Indeed, from an individual member of staff’s perspective there is little difference between not being included in the REF and being included but with no assigned outputs. The report suggests that some staff should be allowed to submit six publications, with the implication that it is because they have outstanding publications, but this requires other staff to have their submission reduced. ‘REF heroes’ require ‘REF zeros’. Now, selectivity is not based on the failure to meet the requirement to have four outputs, or to have a locally-assessed output score (the selectivity done at REF2014 to enhance the UofA profile), but will be done to allow another colleague to submit more outputs. This is a recipe for discord and division.”
One continual theme in the REF is “reducing the burden” both on institutions and on panel members. Much of the burden comes from things having to be reported to REF in an auditable way. The notion of portability of outputs adds quite an administrative burden as presumably an evidence trail will be required about where, exactly, the bulk of the work was done.
Athene,
You make some good comments here. I thought the Stern review looked pretty constructive. And I’d tend to agree with you about this portability of outputs question. This commoditisation of academics (citations, h-index, “portability”) needs to stop. I can see why ECRs might be worried, since it is perhaps all they’ve seen in their careers so far, but it isn’t healthy. We should be investing in people and ideas for the future, not commodities. As you rightly say, people who succeed in one place often do this when they move. And anyway, if they don’t, then getting a quick ‘hit’ of published outputs for a single REF cycle is cold comfort for hiring the wrong person, possibly for 30 years. It should make institutions think longer term when hiring – a good thing, and probably not bad for ECRs, too.
I suspect that lack of portability could also affect mid career researchers. At my institution (like many) promotion can frequently only be triggered by obtaining a job offer from elsewhere. Being unable to bring REF publications with you will make getting job offers close to the end of the REF cycle much more difficult. So while we would eliminate the superstar transfers, used to game previous REFs, lack of portability could delay people from being promoted at the end of the REF cycle – if you can’t move, what motivation does an institution have to pay you more?
I am also concerned about the detailed rules for portability. For example, I am currently finishing a work which was mostly written in 2011 while I was at a previous institution. It will be on the arXiv this month, but perhaps not published until 2017 (as it will be sent to a journal with a long turn around). If I were to move in October 2016, to whom should this paper belong: my previous institution (where most of it was written), my current institution (where it was finished) or my future institution? Would people still game the system by holding back publications until they had moved to their new institutions?
Lack of Portability would KILL mid-career researchers.
Presently, REF publications are the only chips that an assistant professor holds in the fight to become an associate professor.
The Stern review is written entirely from the perspective of employers.
To minimise “game playing” all that is required is to change the way in which the cut-off point works.