I have sat on enough committees when KPIs (Key Performance Indicators) are discussed, to know that they can be very helpful in moving an agenda forward and identifying where sticking points may be blocking progress. However, they should never be the only goal in any programme of work, nor used slavishly without thought. To take one specific item that arose at a meeting I was at recently: is the number of university spin-outs a good KPI? Or should one only count those that have had £X invested in them, or have more than Y employees. Or survived for more than Z years, licensed their product to a certain number of companies, with a turn-over exceeding some figure….and so on. Creating a spin-out company is, in many ways, the easy bit, but there are all those other metrics that could alternatively be chosen (and no doubt others I haven’t put my finger on) and choosing which to focus on may modify behaviour or lead to different ideas of ‘success’, for the individual or the university.
As criteria were being selected for REF2021 (if that isn’t an indelicate subject to bring up as the next cycle draws to a close), when I was chairing the Interdisciplinary Advisory Panel, we were sent a long list of possible metrics that could be used in the context of interdisciplinary research. After a lengthy debate, we decided that none was really fit for purpose. That put all the emphasis on to the panels to make judgements about quality (as well as whether they were genuinely interdisciplinary), but we were convinced that was the right path to follow. The extensive and thoughtful evidence review, The Metric Tide, more generally aimed at the entire REF process, highlighted many potential dangers in being too dependent on metrics, even though relying on them would undoubtedly have simplified the process and cut costs (but at a cost in a different sense). And as academic readers of this blog will no doubt know, academics (and associated administrative staff) are good at jumping through hoops and complying with the rules of whatever ‘game’ is being played.
Yet not ever using metrics has its own issues. How can one tell if progress is being made? The recent HEPI report, Making Metrics Matter: A more ambitious approach to tackling racial inequity in higher education, highlights why metrics still have a huge role to play in our universities. If one looks at the admission of racialised minority students into higher education, it can be seen great strides have been made. However, if these students then fail to thrive – as one could argue both the attainment gap and completion rates demonstrate – simply counting how many start a course is unhelpful. The reality is, as ever, the right question has to be asked. Too often it isn’t.
To take a different example from education, what is happening in (English) schools? The various measures of success – for a school – derive from exam results. The period Michael Gove was Secretary of State for Education saw rapid changes in what was valued. At the time I was Chair of the Royal Society’s Education Committee. We responded at speed to multiple ‘consultations’, suspecting responses from us and others in the wider community were not going to change anything, not least because the speed of decision-making hinted at no one having time to read what was submitted. The recently-removed idea of the E-Bacc came from this time, and has met with substantial opposition over the years, so its termination will not be much regretted. However, the Gove view was always that it is simply about standards, and that is what should be pushed in schools. I had a conversation once with William Hague (back when I was Master of Churchill College and he’d been giving a talk there), when my attempts to discuss school education with him, simply led to the blanket comment that the standing of English schools had improved in the PISA tables, so we must be doing things right. He wasn’t interested in whether this focus on a ‘knowledge-rich’ education was appropriate for the current world, with Google at children’s finger-tips and the world of work so different from when he was growing up.
Every child achieving and thriving is the current Government’s mantra, and there is absolutely no doubt that they are investing in ways to make that possible, starting with substantial investment into family hubs, early years’ provision, breakfast clubs and so on. But, leaving aside those children who start school not ‘school ready’ and who may struggle to catch up, there are many children for whom the transition to secondary school is difficult and who disengage during their teens when faced with a curriculum that is only directed at exam grades. Teenagers themselves have spelled out how they would like to learn more about financial management for instance. The recent Curriculum and Assessment Review had little to say about this. A recent commentary spells out how the mood music in the profession is shifting away from the Gove ‘traditionalist’ approach, wanting to see less emphasis on the metrics of exam grades. Nor is this simply related to the huge challenge of rising SEND numbers, but rising mental health issues (a problem HE faces too) should tell us all is not well: forcing round pegs into square holes never works. Teachers need time in their working day to consider pupils in the round, according to local circumstances for instance, and not just be bound by centrally-driven metrics that may work for one locality but not another.
KPI’s – definitely a mixed blessing, and to be used with caution not slavishly.
