The Problems of Measurement

How should we measure what is a good outcome from a university education? As David Willetts puts it in his latest report published through the Higher Education Policy Institute (HEPI) ‘The Treasury cast their beady eye over the evidence and worry universities are not delivering the earnings boost which they used to’, demonstrating one sort of answer to the question. Another recent publication, this time from the Institute for Fiscal Studies (IFS), considers social mobility and the lure of London using earnings as the framing of a good outcome for the individual. Is it all about money? Should it be all about the money? It seems to me a very narrow figure of merit for either society or the graduate.  Furthermore, I believe the IFS analysis is oversimplifying the interesting data they have analysed.

The IFS study explores the mobility of those with and without degrees, considered by socio-economic status and ethnic origin, but it is framed as mobility – and this is geographical mobility that is being analysed, not social mobility – as necessarily desirable, as if anyone who doesn’t move has ‘failed’, although that is not their explicit phrasing. I worry about this framing, relevant though it may be to the levelling-up agenda. For instance,

“Places with high average earnings attract graduates through migration. Graduates who grew up in places with low average earnings are more likely to move away.”

They note that non-graduates are less likely to move to London and other large cities than graduates.  Inevitably, these patterns of behaviour also lead to their final conclusion that ‘patterns of mobility exacerbate regional inequality in skills’, so that there is a ‘brain drain from the North and coastal areas’. By framing what is a good outcome of a university education in terms of money, the implicit messaging being given to graduates is ‘it’s all about the cash’. This is a travesty of what a university education should be about. The idea of public good, as opposed to personal gain, cannot be seen in that framing, but becomes relevant when trying to determine what might be done for the areas (Grimsby and Wisbech are pulled out as examples) where the current loss of graduates is marked, and social deprivation is highly visible.

Let us take the specific example of a Muslim woman who wants to stay close to her family and teach in a primary school: she will neither be a high earner nor have demonstrated an appetite for mobility, but she will be of huge benefit to her community, and an excellent role model for younger women. Such a woman should be highly valued, not put down as a statistic of someone who didn’t aspire to geographical mobility. (I wonder how the likely increasing trend to working from home, possibly home being located far distant from the location of the employer, will skew future analyses.) Interestingly, the IFS findings show that Asian women who do move earn less than those who stay put (in contrast to every other group), and that young adults of Indian and Pakistani heritage are significantly less likely to have moved by age 27 than their white peers of otherwise similar backgrounds.

Income can be measured.  Value to a local community is much harder to quantify, but it is still value. It is obvious that high paying jobs are often in metropolitan areas which will therefore act as an attractor to many, and if income is used as the only figure of merit, then the metropolitan areas look ‘good’. It becomes a self-fulfilling prophesy that the well-heeled parts of the country are the ones to which graduates may aspire to move, which will definitely exacerbate inequality.

However, here as in so many instances, metrics should be used with care. Willetts highlights the dangers of using, as a proposed criterion for the OfS (Office for Students), graduate outcomes a mere fifteen months after graduation as a means to determine university performance, a time when many (if not most) graduates are still finding their feet in their careers. Fifteen months is far too soon, as he makes clear, and it still relies on using earnings as the figure of merit, this time to score university courses. As so many commentators have pointed out, such a metric will necessarily imply that many courses are inherently seen as low-value; not necessarily the STEM courses I typically consider on this blog, but courses in music and the arts, for instance, and those wishing to become nurses, work in the charity sector (a popular choice for Cambridge graduates I understand) or who aspire to be that primary school teacher I mentioned above.

Willetts is a great believer in universities of all complexions and wishes to see less distinction made between academic and vocational pathways, as well as less between those institutions providing either. He has, it would seem, little time for his alma mater of Oxford, or indeed mine of Cambridge, accusing them of snobbishness, in large part because they don’t teach non-academic courses (while, a little confusingly, simultaneously pointing out that engineering and law are vocational, courses very much taught here). He highlights the potential benefits to the local economy if new universities were to open up in towns from Wigan to Peterborough. Such benefits would pertain at least as much to workers in low-paid jobs as to the graduates themselves. However, the IFS study would imply such graduates would, possibly should, then migrate to the bright lights of London or Manchester, removing the possible gains to the community.

What about Oldham? I highlight this town, sitting within Greater Manchester, because it is in the process of carrying out an economic review, led by Alun Francis, the head of the local Further Education college, Oldham College. (Willetts sees FE Colleges as a great undervalued resource in the education landscape.) What can a run-down old mill town do to improve its economic performance, pushing up median wages from their current miserably low level, and to keep those young people who get qualifications at level 4 and above in the area to boost the economy? As long as the public discourse is framed in ‘geographical mobility is necessarily good’ – to paraphrase the IFS study – or graduate earnings are the only measure of the value of further and higher education, as the Department for Education appears to want OfS to think about the matter, we will not get anything approaching levelling up, because those who can, will migrate to the city lights, and those who advise them will encourage them to do so.

Apologies for early readers of this post, who will have seen that, for some reason, the text appeared multiple times. Thank you to those who drew this to my attention!

 

 

This entry was posted in Careers, Education and tagged , , , . Bookmark the permalink.