The Flattened Curve

The lockdown might have flattened the curve of infection and death, but it has also flattened the curve and swell of life. Existence has shrunk to fit within four walls; life ‘outside’ has largely been compressed within the flat rectangles of my phone and computer screens.

Life in lockdown

You might think that, as an academic, I would revel in the life of the mind, the kind many of us now have to accept whether we like it or not. And I do, normally. It has been no great inconvenience to me to continue working from home, but while I still go to work every day, I am no longer going out to work. The daily plunge into the noise and grit of London amid the press of my fellow commuters, the cut and thrust of meetings and lectures, the faces and places of the university campus – all these things have fallen away. Instead each morning after breakfast I climb the stairs to our small spare bedroom. A home office that I would use occasionally on the weekend has become my daily retreat, wherein I commit to a timetable of days that feel long and weeks that seem short.

Outside of work I have time for reading, for dipping into box sets on TV, and for the occasional ‘trip’ to the theatre. My wife and I have enjoyed the National Theatre’s online programme, sometimes joining with friends for drinks over Zoom before the show and at the interval. We keep in touch with family and friends through messaging and video calls. It is odd to think how a technology that only a few years ago seemed the stuff of science fiction has become such a commonplace. That facility is not available to my parents, however, who are secluded in a nursing home and a fog of fading memories; to them I write letters.

So, there is no shortage of activity and yet it feels like inactivity. All attempts to look outward seem to direct the gaze inward. We need the colour and commotion of the real world to move us out of ourselves. Without it the palette of life shades into monotony. When at the start of each video call people ask how it’s going, I can only think to say how boring life feels.

Metropolis on the horizon

London – a mirage on the far horizon

And yet, I am among the lucky ones. Across the country the virus’s pitiless bill of death and grief has had to be paid many thousands of times over, but it has not yet landed at my door. For many more people, even if death has not wrenched away family and friends, the lockdown has cost jobs and livelihoods. Although the government’s support schemes offer them some mitigation, the fears of an uncertain future must surely tighten the strains of confinement and privation. I have some small share in that uncertainty but can be reasonably confident that my university job is safe. I have family around me – there are four of us in this household and we rub along well for the most part; the wrinkles that trip us up from time to time are only small.

Even so, I don’t really know just how lucky I am. Social distancing extends far beyond the two-metre limit imposed on encounters in the park or at the supermarket and has long been a feature of our segregating society. For those of us who enjoy a comfortable existence, it is an effort to access the crueller experiences of people who do not. That effort is harder still when self-isolation is national policy rather than a habit acquired in the rush through modern life. Which is not to say that it is impossible. There is no shortage of news or commentary and I do not intend to add much to the latter here, not today at any rate. But the coronavirus pandemic has unpicked our pretensions at equality, disproportionately targeting black or minority ethnic people, shredding the lives workers for whom a full-time job is not enough to pay the bills, stalling the lives of the young, and dumping the burden of care for children and the elderly where society still thinks is its natural home: the backs of women.

Already I see that my attempts to look beyond my own circumstances are driving me to broad generalisations. I can’t speak for everyone or every situation and it is perhaps foolish to try. For the time being, life feels suspended and detached. It might provide time for reflection, but I remain unsure of my grasp. For now, it must suffice to try to keep the bigger picture in view and seek out the stories of others. The pandemic has separated us out, but it has also heightened awareness of our common humanity. This virus crosses borders far more easily than our politics has ever done. Even if some leaders are using it to stoke the flames of nationalism, I hold to the hope that many among us will see the hiatus as a time of preparation, so that when the moment comes to re-engage – as surely it must – it will be with renewed new vigour and purpose.

 

Posted in Science | Comments Off on The Flattened Curve

The still unsustainable goal of university ranking

The new and improved Times Higher Education (THE) Impact Rankings 2020 were published this week with as much online fanfare as THE could muster. Unfortunately, they are not improved enough.

Screenshot of Duncan Ivison's article, which is surrounded by an advert bragging about Sydney University's ranking

Sydney University’s Duncan Ivison makes case for impact rankings. And then you notice the advert.

The Impact Rankings score participating universities on how well their activities contribute to the UN Sustainable Development Goals (SDGs), which range across issues such as poverty, gender equality, climate action, health and well-being, peace and justice. Although the compilation of the rankings is primarily motivated as a way to celebrate the real-world impact of what many universities do, a noble aspiration that I applaud, the core methodology remains unfit for purpose. At its centre, as with almost all rankings, there is an intellectual hollowness that undermines the whole project, and it is disappointing to see that the THE has yet to take responsibility for their methodological shortcomings. It is even more disappointing to see some universities abandon critical thinking in their rush to embrace the results.

In an article published just before the Impact rankings announced, Duncan Ivison, the Deputy Vince-Chancellor for Research at the University of Sydney, welcomed the increased focus on university activities that advance the SDGs. He notes that:

“there is now a remarkable global consensus on the importance of the 17 domains identified by the SDGs and the challenges we face in ensuring the well-being of our people and our planet. The framework provides a way for governments, industry, civil society and universities to consider how they can contribute to addressing these global challenges.”

I agree with Ivison that the attention brought to these important components of university missions is a useful contribution to a much wider debate within society about what governments, and the publics they represent, should expect of their institutions of higher learning.

Where I part company with Ivison is in the tenor of his caveats about the ranking process. His notes of caution are too lukewarm. He warns that too narrow a focus on impact (a common, though not entirely unreasonable, preoccupation of governments) risks undermining investment in curiosity-driven research that can have major but unanticipated impacts, and closes by conceding: “There are limits to what universities can do and the SDGs don’t capture everything about the impact of our research.”

They sure don’t. But the problems run deeper. When the 2019 Impact Rankings were published, I wrote a detailed critique that I think stands the test of time, so I won’t repeat the argument in detail. In essence, I identified three major problems of arbitrariness and incompleteness within the THE’s ranking methodology:

  1. Six of the seventeen SDGs were not included.
  2. The rankings are based on an overall score made up of four components: the score for SDG17 (‘Partnerships for the goals’ – a measure of collaboration and promotion of best practice in work towards SDGs) and the three highest scores that the university is awarded for any other SDG. The pragmatism in this approach is obvious, but it means that the overall scores are incommensurable – the THE is not comparing like with like.
  3. The score for each SDG is made up of an arbitrarily weighted tally of very different activity indicators (e.g. research, student numbers, policy development) that, as well as providing only approximate and incomplete evaluations of a rich spectrum of endeavour, are – again – incommensurable. I dissected the problematic nature of these tallies for SDG3 (Good health and well-being) and SDG12 (Responsible consumption and production) last year.

Only one of these issues has been addressed in the latest ranking. All 17 SDGs are now included, but the largest flaws in the process are untouched. As a result, the THE clings to a methodology that despite taking insufficient account of the false precision and the uncertainties introduced by the proxy nature of the indicators used to ‘measure’ actual performance, still claims to be able to distinguish universities on scores that differ by 0.1%. It is laughable to claim this level of precision. It is to universities’ discredit that they go along.

But there is an even more serious problem. Not one particle of the work of universities towards the SDGs, trumpeted so much by the Times Higher, counts towards their score in their World University Rankings, which the THE considers to be their ‘flagship analysis, […] the definitive list of the top universities globally’. These are the rankings that increasingly drive institutional behaviour – and competition between them. Each year’s announcement of the THE’s World Rankings is festooned with stories about this or that university rising or falling, winning or losing in the race to the top. The precise opposite of the collaboration that Ivison, waxing lyrical about the Impact rankings, points to as necessary for humanity to face our global challenges. When push comes to shove for the rankings that matter, the THE assigns impact a weight of precisely zero.

Cynically, one might suppose that part of the rationale for creating their impact rankings is to divert attention from the growing chorus of criticism of university rankings. That cynicism draws strength from the continued lack of response or action from rankers to valid criticisms of their methods. As I wrote last year, “Rankers need to embrace the full complexity and diversity of what universities do, while at the same time being more open about the uncertainties in the measurements and the incompleteness of their analyses.”

But I am not given to cynicism. I still believe that, at heart, many of the people involved in rankings work want the best for our universities and our world. The THE deserves credit at least for expanding the range of university activities that are publicly rated and they are by no means the only ranker that needs to engage with the sector with a great deal more rigour. But their present methods are still unsustainable, and my offer to work together to improve them stands.

La-Normale

And indeed, the moment is opportune. Stranded and stalled as we all are by the COVID19 lockdown, we have a chance to reflect and rethink. The task before us aligns with broader economic and societal concerns that have been brought painfully into focus by our present predicament. Leading economists such as Paul Johnson and Mariana Mazzucato, and even the Financial Times are calling for fundamental changes to the workings of capital and the social contract. Their calls echo the longer-standing appeals of philosopher Michael Sandel and entrepreneur and writer, Margaret Heffernan, to recognise that our obsession with numbers, with performance, with efficiency, with the bottom line, and with ranking is obscuring the thing that really matters – the quality of people’s lives.

 

Posted in Research Assessment, Scientific Life | Comments Off on The still unsustainable goal of university ranking

Three weeks

Just three weeks ago, on eve of the weekend, my wife and I met an old friend for dinner at a restaurant in Southwark. Even then, the most normal things in the world were beginning to feel risky.

Two metres apart

Our friend works for Public Health England, but even if he hadn’t, the conversation would still probably have locked onto the coronavirus pandemic. We could feel it coming – my university had already decided to move all teaching online – and could see it in the news reports from Italy, France and Spain; but somehow we still couldn’t bring ourselves to believe it was part of our lives. When we’d sat down to eat there were only a few scattered diners in the restaurant. By the time we got up to say our goodbyes, the place was full. 

For me, the pandemic arrived over the weekend when another friend, a virologist in Wuhan, emailed to express his shock and dismay that the UK government had opted to rely on a herd immunity strategy to get the country to the other side of the crisis. He painted a vivid and chilling picture of the severity of the disease and the high mortality that would result when health services became overwhelmed. The British policy was ‘cold-blooded and brutal’, he wrote. I relayed the information to my friend at PHE and to a colleague at Imperial College who is heavily involved in coronavirus work, though by then it looks as if the government had already decided to change tack.

On the Monday (16th March) I went in work but not as normal. Fortunately, I was able to go in late and head home early, avoiding the most crowded times on the train and the tube. I returned with everything I would need to work from home and sat in silence with my wife on the sofa to watch the Prime Minister announce the first stage of the UK lockdown. Then it was real. 

But it would take another week for the PM to shut pubs, restaurants and shops, and insist on full social distancing. In his sombre broadcast on the evening of Monday 23rd March, Johnson looked earnest for the first time in his political career. He is a ruthlessly ambitious and deeply unserious man, but on this occasion even ‘Boris’ had the wit not brush away the threat of the situation as the fear-mongering of doomsters and gloomsters.

So here I am, three weeks in to the new normal. I gave no promises when I started writing this blog back in 2008 that I would have anything particularly insightful to say and I feel pretty much the same today. For me the change has been significant but hardly debilitating. I can get on with my job and have no immediate fear that it will be snatched away from me. My working and social lives have contracted to the size of a laptop screen, but everyone in my household remains healthy; we have enough to eat and drink, and are lucky enough to have a garden.

I know it is not the same for so many others – far from it – but it is hard to get a grasp of that. The new normal has brought with it a new kind of distancing from the world, in spite of the endless streams of news and numbers about the the pandemic’s march of misery. That can hardly be healthy. I have signed up to volunteer for the NHS; however, I was slow to do so and have heard nothing back. I am still doing full days at my job but feel strangely disconnected from it. I can’t make out if I’m making any real headway. My mood waxes and wanes. I have enjoyed as much as anyone the funnier side of the lockdown that peppers my twitter feed with jokes and silly videos, but there are the times when levity just seems wrong. Who to turn to then to figure that out? I am fortunate to have my family around me of course, but still miss the stir of friends and colleagues. I guess that is why I thought I would go back to my blog, even if tonight I have no answers and little idea of where we’re headed.

 

Posted in Science | 2 Comments

This is not my Brexit day

It is 31st January 2020 and as of 11 pm tonight the UK will no longer be a member of the European Union. We have arrived at Brexit day.

U2 and the EU

But this is not my Brexit. I did not want it. I did not vote for it. I argued against it with as much reason and reasonableness as I could muster. Throughout the referendum campaign, I never heard a political, economic or democratic argument sufficiently rooted in reality to mount a credible case for leaving the EU. I’ve still not heard one. Brexit will not satisfy those who voted Leave because of political and economic neglect. The solution to those problems lies elsewhere.

This is not my Brexit. In the immediate aftermath of the vote on 23rd June 2016, I was angry and upset. I had thought that even if the vote went the wrong way, an enormous dose of British pragmatism would be brought to bear to smooth the transition, but I soon learned how badly I had mis-judged the situation. Theresa May’s premiership will be remembered, if at all, for the vacuous phrase ‘Brexit means Brexit’, the cold hardening of her anti-immigrant instincts, and her blindness to the fissures that have opened up across the UK, which left her incapable of learning the political lessons of the Brexit vote.

This is not my Brexit. The arrival at No. 10 of Boris Johnson – mendacity made flesh – is a depressing confirmation that Leave campaign’s “dishonesty on an industrial scale”, in Michael Dougan’s famous phrase, has become the new normal in British politics. Johnson may have secured a parliamentary majority sufficient to deliver Brexit, but did so with a minority of the votes cast. So far he has offered little but his trademark bluster to paper over the divisions within the country. Brexit remains a largely English ‘victory’ and a hollow one at that. Somehow we have to find a way to re-unite as a country but also as a people that looks out to the world with generosity, honesty and humility. I see none of these qualities in our current prime minister.

This is not my Brexit. Johnson has played recklessly with nationalist sentiment, inflated the fantasy of British exceptionalism, and stood by while some of his fellow-travellers stirred xenophobic bile into the national discourse. The result has been chilling for millions of Europeans who made Britain a home where they hoped to build a future. The country’s humanity has been diminished alongside its international standing.

This is not my Brexit and this is not my Brexit day. I cling to hope but will mourn the severance. If I have learned one thing in the past three and a half years, it is that nationalism is a cancer that humanity may not survive. It is the root of racism. It thrives on separation when the world desperately needs to find its common humanity. By the happy accident of being born in Ireland, I have been able to apply for an Irish passport. I will not revel in my Irishness, but in the fact that I remain a citizen of something greater – the EU. Even then, I am no longer satisfied. I look forward to the day when my documentation proclaims me what I truly wish to be: a citizen of the world.

 

Posted in Science & Politics | 9 Comments

2019 in 31 photographs

My computer tells me I took over 3,700 photographs in 2019. Yikes!

Photos of 2019

However, I have winnowed them down to just 31, should you care to take a look. I have been fortunate this year to travel far and wide – or should I say reckless? Either way, if you click on the image above, it will take you to the album on flickr.

 

Posted in Scientific Life, Travel | 2 Comments

Books read in 2019

In a kinder, happier age, when I used to write regularly for the Guardian’s science blog network, I would post summaries of the books I had read at the end of each year. Since the network closed in 2018 I have rather lost the habit. Looking back a the list of titles I got though in 2019, I realise how much I share with Robin Ince the problem of retention. I can only marvel at those who seem to be able to analyse the plotlines and arguments of books that they have read months and years ago. My recollections are more impressionistic. I should take more notes.

Superior - by Angela Saini

But let me at least share with you my impressions, such as they are. Here, in the order that they were read, is my year’s worth of books.

  1. A Bigger Prize: why no one wins unless everyone wins, Margaret Heffernan
  2. The Hunt for Vulcan, Tom Levenson
  3. Science 3.0, Frank Miedema
  4. The Good Immigrant, Various
  5. Heroic Failure: Brexit and the politics of pain, Fintan O’Toole
  6. I’m a joke and so are you, Robin Ince
  7. Utopia for Realists, and how we can get there, Rutger Bregman
  8. The Tyranny of Metrics, Jerry Muller
  9. Invisible Women, Caroline Criado Perez
  10. Superior: the return of race science, Angela Saini
  11. The New Silk Roads: the present and future of the world. Peter Frankopan
  12. Made to Stick, Chad and Dan Heath
  13. Natives: race and class in the ruins of empire, Akala
  14. Intelligence, Stuart Ritchie
  15. The Lagoon, Armand Marie Leroi
  16. The Gene Machine, Venki Ramakrishnan
  17. Beyond Weird, Philip Ball
  18. Adventures in the Screen Trade, William Goldman
  19. Red Notice, Bill Browder
  20. Manhattan Transfer, John Dos Passos
  21. The Goldfinch, Donna Tartt
  22. Why do so many incompetent men become leaders? Tomas Chamorro-Premuzic
  23. Generous Thinking: A radical approach to saving the university, Kathleen Fitzpatrick
  24. Translations, Brian Friel

It has been another bad year for reading fiction – just two novels and one play. Co-incidentally, both novels were largely set in New York, separated by about a century. While I disliked the fragmented, panopticon storytelling in Dos Passos’s Manhattan Transfer, Tartt drew me page by page into her intricate web of love and loss. The movie version of The Goldfinch was a bland disappointment, but Fabritius’s original painting, which I finally saw in the Mauritshaus in The Hague in October, radiated complex, new life.

The Goldfinch - in the Mauritshaus, The Hague

It’s also not been a great year for reading books by women – just five-and-a-half* out of twenty-four. That said, it’s arguably also been a good year since three of my top four picks of 2019 were by women: Angela Saini’s Superior, a magnificent and powerful exposition of the science of race and the racism of science; Caroline Criado Perez’s brilliantly illuminating Invisible Women – which does that rare thing of making you see the world anew; and Margaret Heffernan’s A Bigger Prize, a lucid and disarming examination of the dark side of competition.

The fourth spot goes to The Lagoon, by my Imperial College colleague Armand Leroi, whose affectionate and deeply informed guide to Aristotle’s science I found wonderfully companionable. (I reviewed Superior for the Cosmic Shambles blog back in July and would agree with every word of Adam Rutherford’s assessment of The Lagoon. I’m afraid I only managed to tweet about Heffernan’s and Criado Perez’s books).

Beyond that, I enjoyed to a greater or lesser extent every book I read. Levenson’s The Hunt for Vulcan was a nicely wrought tale of how science actually works, while O’Toole’s Heroic Failure helped me a little more to deal with the pain of Brexit. In Natives Akala provided a personal and political examination of racism in the UK; along with The Good Immigrant it was a chance to listen to voices that are still too often unheard.

Peter Frankopan’s The New Silk Roads returned in the 21st Century to the ground he had covered historically and magisterially in The Silk Roads (which I read in 2017), while Philip Ball’s Beyond Weird did not disappoint; if it still left me somewhat baffled, that is only because I raced through it on my summer holiday. When relaxing I should really stick to easier reads. On that same holiday I enjoyed William Goldman’s gossipy Adventures in the Screen Trade, a behind-the-scenes look at the movie business and was electrified by Bill Browder’ terrifying story of corruption and murder in Putin’s Russia (Red Notice).

This past week I have finished off Kathleen Patrick’s Generous Thinking, which touches on the over-metricisation of academic life, a theme also covered in Muller’s The Tyranny of Metrics and Bregman’s Utopia for Realists. I have found inspirational material in all three books for my roles championing equality, diversity and inclusion at my university and as chair of DORA, an organisation campaigning to reform research evaluation. All three authors are searching for ways to repair the damage done to ideas of the ‘public good’ by the relentless machinations of the market**. I will surely draw on their wisdom in the year to come.

Translations - the stage at the National Theatre

And finally, I went back to Brian Friel’s Translations, a play I first saw in Boston in the mid-1990s and then again a couple of weeks ago at the National Theatre in London. Although I felt that the drama was ill served by the staging in the most recent production, I was sufficiently stirred to revisit a text that is a richly layered meditation on language, history, memory, and belonging.

None of the above does proper justice to the books or their authors but I hope it might light a few flickers of interest for some people. I have already embarked upon my first title for 2020, Oliver Morton’s The Moon: a history of the future, which even in the first chapter is proving to be poetically entrancing.

 

*About half the authors in Nikesh Shuklah’s edited collection, The Good Immigrant, are women.

**While I’m on the subject of the ‘public good’, let me also recommend Michael Sandel’s lecture, A New Politics of Hope, a humane and much-needed response to the populism that seems to have overtaken the UK and the US.

Posted in Book Review | Comments Off on Books read in 2019

My carbon bootprint

What was your carbon footprint for 2019? Mine was more of a bootprint, almost entirely because of flying.

Airplane - B&W

International travel has long been considered one of the perks of academic life, something that lifted the job out of the ordinary and cemented our membership of a trans-national community of scientists and scholars. Over the years I have travelled to Grenoble and Hamburg for experiments, to Tokyo and Honolulu for meetings with collaborators, and to places like Beijing, Santiago (in Chile) and New York for scientific conferences.

For much of that time, though I may have travelled far, the trips were relatively infrequent. Some years, particularly early in my research career, I did not venture beyond these shores. Now that I have clocked up a few more years and moved into management and policy work, I have also been clocking up the air-miles.

2019 was my worst year yet. By my count, I gave at least 26 invited talks over the last twelve months, 16 of them outside the UK. I travelled abroad on five further occasions on university business and to attend grant panel and advisory board meetings. I also flew to Ireland six times to visit my elderly parents, and my wife and I treated ourselves to a trip to Marrakech at Easter. As well as keeping count of the number of journeys, I was moved this year to tot up the amount of carbon dioxide added to the atmosphere by my travels. The results are pretty shocking.

There are a variety of online tools with to help you work out the impact of your air miles,  which (from my limited testing) give similar results. More or less at random I used the calculator at www.carbonfootprint.com, and got the results below.

Carbon Footprint from flying in 2019

These numbers are a bit squishy because there are factors that are tricky to take into account. For example, long-haul is ‘better’ than short-haul, because taxiing and take-off use proportionately more energy; and newer planes are more energy-efficient. The tally also depends on how many of the seats on the plane are occupied and where you are sitting: fuller planes and economy seats that take up less room are less damaging to the environment per passenger mile. You can also choose in this calculator (as I have done) to take account of radiative forcing, the increased effect on global warming of carbon emissions at high altitude – which increases your total by a factor of about 1.9.

Even so, I will assume that the resulting tally is reasonably robust and mine is very large. By flying for work I effectively added 20.6 tonnes of CO2 into the atmosphere and a further 4.0 tonnes because of family visits and that trip to Marrakech, making a grand total of 24.6 tonnes. That compares to the annual total for each UK citizen (averaged over all domestic, industrial and commercial activities, including transport) of about 5.6 tonnes. That number is 8.6 for the average EU citizen, 16.1 in the USA, 2.4 in Brazil, 1.9 in Morocco and 0.6 in Nigeria (see Wikipedia for the full list).

Globally, transport accounts for about 22% of carbon dioxide emissions. Of the various modes of transport flying is the most damaging (closely followed by cars with a single passenger). As someone who commutes to work by train and tube, I don’t do many miles by car each year, but clearly my flying habits are problematic. Not only am I among the 15% of British people who take 70% of all domestic and international flights, I am in the 1% that takes over twelve flights a year.

Although almost all of my trips in 2019 have been due to my work to promote equality, diversity and inclusion in higher education, and to advance DORA’s campaign to reform research assessment in the academy, my efforts to make the world a better place are undermining climate campaigners who are trying – increasingly desperately – to save the planet from harm. I might allow myself some mitigation because of the nature of my work, but there’s an obvious dilemma here and I need to cut back on my travelling. But how?

There are some practical measures I can take. I already make pretty good use of the train where the time-penalty is minimal to non-existent – always within the ‘mainland’ UK and for trips to Paris, Brussels and Amsterdam. With better planning (and websites like www.seat61.com), I should be able to extend the range of my train-travel but there are practical limits on that. When invitations come in, I will also think harder about whether I should accept and ask more pointed questions about the purpose of the meeting and the size of the audience. I can also consider alternatives to me taking the trip. For DORA-related invitations, for example, I can do more to call on members of our steering committee and international advisory board who may be geographically closer to the host (I have already done this for a meeting I was invited to in New York next summer). And although giving a talk by video-link is still very much a second-best option, both because of the unreliability of the technology and the loss of face-to-face contact that is still so enriching in this networked age, I’ll start seriously considering that as an option as well.

These are small steps – probably too small. I don’t imagine for a moment that Greta Thunberg will be impressed. It is at least a start, but to help me go further – and perhaps be more radical – maybe you can leave a comment to say what you are doing to reduce your carbon bootprint?

 

Posted in Science, Scientific Life, Travel | 10 Comments

Time for reflection

I think of Sunday as the last day of the week, not the first. Today, at the end of a hard week on political and personal fronts (though why the political and personal should be seen as separate I am not sure), I flew to Ireland to visit my parents. I am writing this on the plane back to London.

Flying to Ireland

This week’s election has come and gone and delivered a result that leaves the country in a deeply worrisome state. The Conservative victory was built on simplistic sloganeering, evasion of hard questioning and the scattering of misleading claims across social media. It’s a strategy that can win elections but which, if a divided and mis-managed opposition can get its act together, will unravel as soon as reality bites. 

There will surely be a heavy reckoning, but it may take longer than many of us might hope for the hard hand of consequence to be felt. In the meantime, the Conservative manifesto and Johnson’s autocratic instincts present clear and present dangers to core national institutions such as parliament, the judiciary and the BBC and Channel 4. 

Prime Minister Johnson may have warm words now about the need to “let the healing begin” but this is a man whose defining feature is a lack of care for the truth. There is no point in listening to what he says. His government must be judged solely on its actions. Perhaps, perhaps (why is hope so slow to die?) because of the surprising Conservative victories in traditional Labour strongholds in Wales and the north of England, he means what he says about investing in the NHS? My hard head resists. So let us keep count of the number of nurses and the actual levels of investment. 

As far as science is concerned, there may be a promise to double the R&D budget, but let us keep watch on what really happens. Science is Vital may have hung up its boots, but I hope CaSE will continue to keep government R&D policy under the microscope. 

Johnson’s determination to wrench the UK out of the EU, on a timetable that no-one who understands international trade relationships thinks credible, will surely damage our participation in multilateral funding schemes that are the envy of the world. The groundwork for that was laid by the blinkered and venal anti-internationalism of Theresa May’s tenure; the Daily Mail might have crowed, but our foreign colleagues were left scratching their heads at Britain’s loss of contact with the realpolitik beyond our shores. Johnson might be cut from more liberal cloth but we can take little solace from his advisor Dominic Cummings’ professions of interest in research. For all the words that he has expended in his blog posts on the topic, Cummings has yet to steer his boss to a plan for R&D investment that makes scientific sense; or one that attempts to bridge the divides exposed by Brexit between those who are comfortable with Britain’s place in Europe and those who are not. 

It is all so horribly depressing – government by mendacity and an opposition in disarray. But we cannot afford to lose hope. Life must stumble on and somehow we must find a way to reconnect politics with integrity and with what’s real. 

In the meantime, at the personal level, life cannot help but stumble on. Sometimes it stumbles to a halt. To the dreadful election results of last week I have to add news of the death of my sister-in-law’s father – a kinder Englishman you could not wish to meet – and the passing of the father of a good friend in France. The first was expected and feared, the second a saddening shock. And then today I visited my own parents – ailing, confused and upset. The candle gutters; theirs has been a hard year. I look at them, once so strong, and fear selfishly for my own future.

Stage set for Translations

The week was leavened by my birthday, celebrated with our grown-up children at a production of Brian Friel’s Translations which, though a tad disappointing on some technicalities of the staging, still had the power to stir this Irish heart.

So I’m not done yet. While I still have my wits about me, I can hope for a better future and keep on trying to do something about it. 

 

Posted in Science, Science & Politics, Scientific Life | 1 Comment

Berlin Debate – Who owns science?

Yesterday I had the privilege of participating in the 14th Berlin Debate on Science and Science Policy, which is hosted by the Robert Bosch Foundation. This year the invited participants discussed “Who owns science? Reshaping the Scientific Value Chain in the 21st Century“. There was a lot to unpack in that topic but for my formal contribution as chair of DORA (a six-minute opening statement), I took the opportunity to explore the links between open science, research evaluation and inclusion. My statement is reproduced below: 

I have been asked to look at one of the key challenges on the road to Open Access and Open Science: How to secure and measure quality if we stop relying on the Impact Factor (and the peer review-systems) of prestigious, but paywalled journals. 

The first part of that question, how do we secure quality, presupposes that the present system of scholarly publishing does a good job of quality control. Now I am not a fan of talk of a “reproducibility crisis” or of proclamations that “science is broken”; but clearly, despite the fact that many authors, reviewers, editors and journals are doing a good job, there are problems. 

Our systems of research evaluation have become over-metricised. In particular, as we all know, we have an over-developed reliance on journal prestige – most often signified by the impact factor – as a measure of achievement. 

This means that the dual functions of publication, first to share new discoveries and insights, and second to establish a reputation on which to build a career, have become unbalanced. The chase for a metricised reputation panders to selfish instincts and stokes hyper-competition by focusing on objects (papers and grants), rather than on people. 

It has led to numerous perverse effects that undermine science: 

  • problems with the reliability of the scientific record; 
  • increased retractions, which undermine public trust;
  • delayed publication which, if the public was aware of it, would further undermine trust; 
  • we have a very patchy record on data and reagent sharing; 
  • and a publication system that is still largely closed, not just to the public but to many academics.  

Our obsession in research evaluation with metrics such as impact factors is an unhealthy breach of science’s social contract and increasingly puts scientists in conflict with the values that many claim underpin science. 

Those values are rooted in the moral purpose of science, which is to make the world a better place – for everyone. They have emerged over centuries of continuous discourse. I don’t have time to excavate the history in detail so let me be content with a single footnote from Robert Merton’s famous 1942 paper on science and democracy, where he highlights remarks made by the 18th century French entomologist and thinker René Antoine Ferchault de Réaumur. Writing about the moral compulsion to publish one’s research, and while acknowledging national obligations, Réaumur ultimately concluded that “those who work to advance science and the arts should consider themselves citizens of the world.” 

To me this is a 300-year old vision of open science. And it is my conviction that open science offers a route out of many of our present difficulties. It provides opportunities to reconnect with our most fundamental values, not just as scientists, but as human beings. 

To be sure, there are some practical difficulties to be overcome in moving toward open science practices (inequities of cost, for example, that I hope we might explore in the next session), but there are no fundamental impediments and no serious threats to the quality of our research work. In fact, in very many ways, an open approach will improve our work because it demands that we expand our view of the qualities of research that matter and that should be evaluated. 

The rise of preprints, for example, makes science faster; and because they present results independently of any journal brand, they provide a healthy focus on the quality of the content. Their openness (as with open access papers) means that they are available for the widest possible scrutiny – an incentive to authors to get things right – and for the widest possible re-use. Preprints also appear to foster commentary in ways that peer reviewed publications have struggled to do and this is helping to pave the way for experiments in open peer review (a practice that, if adopted universally, would kill off predatory open access journals). 

Just as importantly, the rise of open science raises the question of why academic papers should be available to all. It is not just, as many governments reasonably contend, that the public has the right to see what they have paid for, and to demand maximal returns on its investments, but also because there are many stakeholders with a genuine interest: patient advocacy groups, environmental campaigners, citizen scientists – indeed any citizen concerned about the direction of travel of the modern world. 

The dialogue engendered by open science raises the question of who should shape science. This is not just about engaging with the public, enabling them to have a say in setting the research agenda*. It is also, crucially, about taking a closer look at the fact that in no country that I can think of is the academy representative of the population at large. Women and ethnic minorities, among other groups, are chronically under-represented. This is in part a result of our present system of evaluation: metrics and rankings too easily sustain the status quo. How can an academy that is not truly global claim to be able to identify the world’s most pressing problems? 

Open science compels us to re-examine what it is that we – the diverse citizens of the world – want from science, and to figure out how we are going to evaluate that. This is a project that DORA is keen to advance, through advocacy and speechifying, yes, but also by developing and promoting practical solutions.  

To me open science represents hope: a chance to re-negotiate science’s social contract in ways that are more inclusive, more open to public dialogue on the proper balance between academic freedom and responsibility, and more rooted in the moral purpose that so many of us are seeking to give meaning to our lives. 

 

 

Posted in Science | Comments Off on Berlin Debate – Who owns science?

First impressions: the DORA-HHMI meeting on research assessment reform

My feet have hardly touched the ground since I got back from the DORA-HHMI meeting on “Driving Institutional Change for Research Assessment Reform” in Washington DC last week, but I wanted to log a few first impressions. I can’t hope to do it justice but please be reassured that the videos of the plenary session will soon be available, and there will no doubt soon be wiser and more considered ruminations on the conference.

Economist, Paula Stephan

From Monday evening until Wednesday lunchtime (21-23 Oct), we gathered in the plush surroundings of the Howard Hughes Medical Institute headquarters to work through the present challenges in research assessment. I was there in my a capacity as chair of the San Francisco Declaration on Research Assessment (DORA). It was a good size of meeting – about 60 participants; small enough so that you could clock the faces of most of the people there over the three days, but large enough to have a good mix of stakeholders, including vice-presidents for research, faculty deans, researchers, librarians, representatives of learned societies and funders, and the odd economist and behavioural designer.

At meetings on the topic of resesarch assessment, there is a tendency for the discussions to be orbit around the same old problem – the pernicious effects of journal impact factors and the chase for position in university rankings. But I am pleased to report that there was a conscious effort here to direct attention to solutions. This effort was nevertheless rooted in a clearly articulated determination to recall the values that are the foundation of scholarly research, which date back at least to the work of Robert Merton, but are also subject to the questions raised by the open science movement and the increased attention being paid to issues of equality, diversity and inclusion in the academy. These values recall not only our duties within the academy as scholars who subscribe to communitarian principles, but also our responsibilities as mentors, leaders, and managers, and as professionals who started out with a keen sense of wanting to make the world a better place.

The problem with values, as became apparent in our deliberations, is that they can too easily disappear amid the everyday pressures to get the job done. An evaluation system preoccupied impact factors, citations, grant income and league tables soon distracts people and universities from their higher aspirations. How best to keep sight of our larger purpose?

DORA-HHMI - 3

Me, trying to keep up with Frank Miedema…

There was a clear-sighted recognition at the meeting that reform of research evaluation has to be thought through in very practical terms. Values on their own are not enough. For one thing, we need to ensure that university leaders are aware of the problem associated with over-reliance on simple metrics. There is too much of a “Yes, but…” culture that pays lip-service to the problems without tackling the root causes.

As we heard, tackling the root causes requires institutions to have honest conversations about their values – perhaps best conducted at more granular – departmental? – level within the organisation where everyone can be involved. This is key to empowering researchers, from PhD student to professor, to take ownership of the problem of resesarch assessment.

But more than that, we need workable alternatives to the traditional metrics of ‘excellence’, assessment mechanisms that take a broader, deeper view of the aspects of scholarship that matter, that embrace not only the diversity of important outcomes from research (the particular quality of the work, as well as associated outputs such as data, software, trained researchers, etc.), but all the other important activities that academics undertake, such as teaching and mentoring, departmental and disciplinary service, and public and policy engagement. Here there were a number of interesting examples of good practice, including the use of biosketches to capture a more narrative-based summary of individual accomplishments, and structure interview techniques that try to probe beyond the desire to generate papers, for example by testing candidates’ commitment to diversity or their abilities as team leaders. Beyond that we had fascinating and provocative discussions on the economics of science, and the importance of deliberate and systemic approaches to organisational culture change.

DORA-HHMI - 2

Dinner and discussion at the DORA-HHMI meeting

The meeting was intense and invigorating, and many of the discussions were tough. At times they were challenging and uncomfortable – an important path to learning (though everyone still seemed to get along in the evening socials in the bar!). Personally, I am grateful for the insights from vice-presidents of research that I spoke to about the perverse and damaging, but inescapable pressures they come under from university rankings.

The realities of research assessment are harsh and it is clear that, despite lots of interesting and innovative experimentation in this area, we are still figuring out what works best. One of the most important lessons of the meeting is that that’s OK. For sure, we will have to evaluate what reforms work best, but it is good that people are already experimenting and innovating. It will take time to find out how best to escape the perverse effects of over-metricisation, but it is already an important signal to the research community and to society at large that we are trying to do better.

I hope that, like me, most of the meeting participants have returned home not just energised by interactions with a people who committed to change, but equipped with an expanding repertoire of ideas about how to make change happen where it matters: on the ground.

If I have seemed lax in crediting ideas to individuals, it is because much of the meeting was conducted under Chatham House rules which prevent such attribution. But I am more than happy to give credit to those whose written work in this area resonated strongly in the meeting. I list a few of them below as suggested further reading on values and practical approaches to research(er) assessment. I will be happy to add further suggestions – leave a comment or ping me an email.

 

Posted in Science | Comments Off on First impressions: the DORA-HHMI meeting on research assessment reform

I told myself…

I am on holiday – in Mauritius, a tiny tropical island in the Indian Ocean. And when I go on holiday, I make ridiculous plans. 

Mauritius beach

I told myself I’d start running again. I’d get up early every morning, slip out of bed without disturbing my wife and cross the road to the beach. It might be hard at first but I could jog along the compacted sand at the shoreline, the sun rising at my back. It would be good for me. 

I told myself I’d write every day. I’d get up early, slip out of bed without disturbing my wife, make a pot of coffee and sit at the table on the patio overlooking the tropical garden. It would be time just for me; a chance to get back into the groove now that I have a week free from the agitation of work.

I told myself I’d read four books. I packed Philip Ball’s Beyond Weird, William Goldman’s Adventures in the Screen Trade, Bill Browder’s Red Notice, and Manhattan Transfer by John Dos Passos. (Is it that my interests are broad or that I lack focus?)

I told myself the holiday would be a chance to reflect, to dig deeper into some of my preoccupations at work. I would have the time to really think about how to weave together the elitism of a research-intensive university with the equality of opportunity that the integrity and values of the idea of the university would appear to demand. And how to achieve that in the midst of a society that claims to appreciate variety but has never really been comfortable with difference. 

I told myself I would have time to muse on what more I can do to advance the aims of DORA, the Declaration on Research Assessment (which I chair). The argument for reforming research evaluation is technical and slippery. How to inspire and facilitate changes that will truly bring out the best – defined as broadly and meaningfully as we can – in scientists and scholars? 

I told myself I’d avoid emails, and Brexit, and social media. 

Five days into our holiday, I have done almost none of this. No running, no writing beyond this scribbled post, and precious little in the way of deep thought. I have finished Philip Ball’s book on quantum mechanics and started Goldman’s gossipy account of life as a Hollywood screenwriter. I have lacked the discipline to keep away from emails altogether – though keeping in touch while giving myself permission to ignore work has helped my equilibrium. I have tried my best to wrestle free of twitter and Brexit but they do clamour so. 

Mauritius - 2

But I have succeeded in swimming in the turquoise sea that surrounds the island. I have sat on the beach listening to the rumble of the white lines of surf breaking over the distant reef, while the water swishes and laps at my feet. I have felt the bloom of the sun’s heat on my face as the blazing orb emerges from cloud cover, and seen its light turn the sea and sand to silver and gold. I have chatted and reminisced and laughed with my wife and children over cocktails at a beachside bar. I have been surprised to find Jupiter in the southern night sky almost directly overhead. 

So much for plans, but that will do nicely for now. 

 

Posted in Blogging, Scientific Life | 2 Comments

The unsustainable goal of university ranking

Ranking organisations are seeking to diversify the measures use to evaluate universities. But without addressing the fundamental flaws in their methods, they will crush rather than embrace the rich complexity of our institutions of higher learning

THE University Impact Ranking

When the Times Higher Education (THE) released their University Impact Rankings back in April, the idea of scoring institutions on the basis of their contributions to the UN sustainable development goals was hailed by THE’s ranker-in-chief, Phil Baty, as ground-breaking. The new rankings are certainly innovative, but for many academics it seemed like business as usual. Here was yet another way to slice and dice the data on university performance from a global industry that, although non-existent 20 years ago, has become a permanent feature of the higher education landscape. New rankings come so thick and fast these days – Wikipedia lists no fewer than 24 different league tables – that the temptation is to shrug and pass on to the next thing. But that would be a mistake.

On the face of it, THE are to be commended for diversifying the scrutiny of university performance beyond the debatable measures of reputation, research prowess or teaching excellence that are most commonly used to compile rankings. This is a sensible move. But fundamental problems remain which afflict all university league tables and are too often overlooked by student and universities – the major consumers of rankings. Students might be forgiven for not appreciating the uncertainties and incompleteness of the analyses that pull university rankings dangerously close to the realm of fake news, but universities should know better.

I suspect that most do know better but feel compelled nonetheless by market pressures to jostle for position in rankings that originated in the news media and are still granted widespread coverage. I don’t doubt the rankers’ belief in the valuable service they see themselves providing to the sector; and I’m sure they can justify to themselves the hard-nosed, real-world pragmatism brought to the task of scoring university performance, which may even be shared by some university leaders. But we can’t afford to side-step the fractured logic at the heart of league tables – the simplistic quantification and aggregation of incomparable qualities – which tears at the intellectual integrity that is surely still central to the idea of the university. We need to find better ways to set the standards for the multiple and various dimensions of university life so that, as they evolve in a constantly changing world, they can continue to enrich the societies and communities in which they are rooted.

And now perhaps, it is the turn of the rankers to shrug at the ramblings of yet another disgruntled academic. It’s all too easy to criticise and carp, they might say, but what about solutions? Students and governments rightly want to know about standards of performance, so who is going to hold universities to account?

Well here’s an idea: let’s try do it together. Cynics, please look away now. Let’s take at face value the higher aspirations for the health and impact of universities found in the mission statements of rankers and our institutions of higher learning, and proceed in the hope that they might help us jointly to unpick the tangle of political, commercial, and institutional interests that sustains the rankings industry in its present form.

Even if there is the will to do that – and I concede it remains an open question – the technical challenges are formidable. But before we can devise solutions, we need to understand the problem. We can start to get to grips with that by looking at how the THE put together its University Impact Rankings. To their credit, the THE provide a detailed description of how how they go about scoring universities. Let’s look at the big picture first. Here’s the summary of how the overall rankings are compiled:

“We use carefully calibrated indicators to provide comprehensive and balanced comparisons across three broad areas: research, outreach, and stewardship.

“There are 17 UN SDGs and we are evaluating university performance on 11 of them in our first edition of the ranking (click on a category below to view its specific methodology):

Universities can submit data on as many of these SDGs as they are able. Each SDG has a series of metrics that are used to evaluate the performance of the university in that SDG.

Any university that provides data on SDG 17 and at least three other SDGs is included in the overall ranking.

As well as the overall ranking, we also publish the results of each individual SDG in 11 separate tables. This enables us to reward any university that has participated with a ranking position, even if they are not eligible to be in the overall table.

A university’s final score in the overall table is calculated by combining its score in SDG 17 with its top three scores out of the remaining 10 SDGs. SDG 17 accounts for 22% of the overall score, while the other SDGs each carry a weighting of 26%. This means that different universities are scored based on a different set of SDGs, depending on their focus.”

Setting aside the question of what exactly is meant by “carefully calibrated”, the first major problem with this approach is the arbitrariness of the selection of SDGs. Only 11 of the 17 goals are included. The missing SDGs are:

      • SDG 1 – No poverty
      • SDG 2 – Zero hunger
      • SDG 6 – Clean water and sanitation
      • SDG 7 – Affordable and clean energy
      • SDG 14 – Life below water
      • SDG 15 – Life on land

The exclusion of these goals was presumably part of the careful deliberations that went in to the construction of the tables but they are hard to figure out. If your university is doing work on economics or social policy that might tackle poverty, that doesn’t count. The same applies if your institution is active in areas of food or agricultural technology, engineering solutions to the supply of clean water or clean energy, or if it researches environmental issues or biodiversity. Research and education in all of these domains have the potential for world-changing impact but this is effectively given zero weight by the THE. Perhaps there are technical reasons for the omission – a lack of sufficiently relevant indicators or data – but these are not given in the methodology. The aggregate estimation of impact by the THE is therefore incomplete.

The second major problem, which is more serious and more striking, is the non-comparability of the overall scores, since these are based on quantification of different activities at different universities. The impact score of a university is its SDG 17 score added to their highest scores for three other SDGs. This is a pragmatic rather than a scientific choice. And given that what we are talking about here are universities, supposedly society’s stoutest bastions of scholarly critique, it’s an astonishing one. The methodology is even more arbitrary than ‘standard’ league tables that at least score universities on a common set of categories (an approach that still fails to address the issue of assessing overall performance from non-comparable attributes – see below). It’s the equivalent of trying to figure out who is the best at sports by compiling a single league table that ranks footballers, tennis players and racing drivers. There is no intelligent way to do it. In the domain of sport, what’s the harm? The rankings would generate stories that people would have fun debating. But do we not take our universities more seriously? It is hard to escape the suspicion that news values are behind the desire to reduce university performance to a single number. Rankers can and should do better.

Sports stars - who's the best?

Who’s the best? (Original photos via Wikipedia – click image to see links)

Third, even within the scoring mechanism for each of the different SDGs there are issues of arbitrariness and incompleteness. For example, below in outline is how SDG 3 – good health and well-being – is scored. The ranking focuses on a number of disparate but related elements: “universities’ research on the key conditions and diseases that have a disproportionate impact on health outcomes across the world, their support for healthcare professions, and the health of students and staff.” The total score for SDG 3 is totted up from the following components, which track both inputs and outputs:

    • Research on health and well-being (27%)
      • Proportion of research papers that are viewed or downloaded (10%)
      • Proportion of research papers that are cited in clinical guidance (10%)
      • Number of publications (7%)
    • Proportion of health graduates (34.6%)
      • “proportion of graduates who receive a degree associated with a health-related profession out of the institution’s total number of graduates.”
    • Collaborations and health services (38.4%)
      • Collaborations with local or global health institutions to improve health and wellbeing outcomes (8.6%)
      • Outreach programmes in the local community to improve health and wellbeing (8.6%)
      • Free sexual and reproductive health services for students (8.6%)
      • Free mental health support for students and staff (8.6%)
      • Community access to university sports facilities (4%)

As with all such measures, the assigned weightings are arbitrary (and in this case, inexplicably precise). The selected components seem like reasonable targets but what exactly is being measured here? For example, how does one score a university’s provision of “free mental health support for students and staff”? Is there any measure of the standard of service – or the mental health improvement of the recipients? The THE asks for supporting evidence; this is “evaluated against a set of criteria and cross-validated where there is uncertainty.” But the criteria are not given, and although it is good to see acknowledgement of the uncertainties in the information being used to compile scores, there is no estimation or reporting of these uncertainties in the rankings – a long-standing problem that has been raised before but which no ranking organisation has properly addressed.

The scoring of SDG 12 – responsible consumption and production – takes a different approach. Here the focus is on “efficient use of resources and minimising waste”. The total score comprises the following components:

    • Research on responsible consumption and production (27%)
    • Operational measures (26.7%)
      • Policies on ethical sourcing of goods (4.9%)
      • Policies on the appropriate disposal of hazardous waste (4.9%)
      • Policies on minimising waste sent to landfill/maximising recycling (4.9%)
      • Policies on minimising the use of plastics (4.9%)
      • Policies on minimising the use of disposable items (4.9%)
      • Evidence that these policies also apply to outsourced services (1.1%)
      • Evidence that these policies also apply to outsourced suppliers (1.1%)
    • Proportion of recycled waste (27%)
      • Proportion of waste that is recycled (13.5%)
      • Proportion of waste that is not sent to landfill (13.5%)
    • Publication of a sustainability report (19.3%)

Again, while each of the components seems a sensible choice, the score is made up of disparate elements assigned arbitrary weightings. Why should publishing a sustainability report earn almost as much credit as operational measures? There’s no good answer to that – it could be debated endlessly.

As a single estimate of activity in this one area, it may be reasonable to suggest that this method for evaluating progress towards SDG 12 is good enough to win a consensus of sorts among people keen to get on with the job of ensuring that the university uses resources well. But we still have to come back to the fundamental problem with the THE’s overall impact ranking that it depends on an arbitrarily weighted sum of the scores for disparate activities, each of which is the arbitrarily weighted sum of the scores for disparate activities. It is in this unreasonable aggregation that the system falls to illogical pieces.

And remember – these are just the impact rankings. While they might represent a well-intentioned attempt to diversify the benchmarking of university performance and to pull attention away from the narrower focus of standard rankings, I wouldn’t want to see them aggregated into the single score use in the THE’s Global university rankings (and am not aware of any plans to do so). Of course, this means that when the 2020 global rankings are published later this year, much of the attention recently devoted to trying to recognise impact will be lost. That raises the question of how serious the THE is about giving people a truly holistic view of how well the universities of the world are fulfilling their different missions.

There is a better way. That is to include all reasonable estimates of valued university activities – such as their academic and societal impact, and the quality of internal processes such as staff management, resource management, and the various dimensions of student experience, but without aggregating the data. Rankers need to embrace the full complexity and diversity of what universities do, while at the same time being more open about the uncertainties in the measurements and the incompleteness of their analyses.

This disaggregated approach has already been adopted by the Leiden Ranking generated by that university’s Centre for Science and Technology Studies (CWTS). As such, it is an embodiment of the CWTS principles for the responsible design, interpretation and use of university rankings. The Leiden Ranking may more narrowly focused in scope than the suite of rankings generated by the THE, but like them it has started to incorporate elements that go beyond purely academic impact, such as gender balance and institutional commitment to open access. Even with disaggregation, one has to think carefully about institutional and disciplinary contexts to make full use of these data, as Cassidy Sugimoto and Vincent Lariviere have shown in a recent analysis of the gender data on academic authorship. Such care and insight fades from view when university ‘performance’ is boiled down to a single number, and lost from sight altogether amid the headlines of this or that university moving up or down the resultant tables.

CWTS ranking – list view

There are risks in expanding the number of measure of university performance (even if one adheres to the principle of disaggregation). The burden of measurement – which seems to have an inexorable tendency to grow – may soon begin to outweigh the benefits of evaluation. There is something dehumanising in seeking to put a number on every particle of human activity, however worthy the aim; and finding the appropriate balance between quantitative and qualitative modes of evaluation is task that demands constant vigilance and negotiation. But the risks should be mitigated by involving the measured in co-designing the processes of evaluation. The trick is to find effective ways for different stakeholders to do so in good faith.

I believe Phil Baty is sincere when he says “Universities make the world a better place in so many different ways. @timeshighered is delighted to champion that work…” That is a vision shared by many who lead and work in universities – and one that surely has an enduring appeal to students. But the credibility of that vision depends critically on our ability to develop a collaborative approach to delivering it. If you read the rest of Baty’s tweet it says “…with the new University Impact Rankings, out in April #THEglobalpact #SDGs”. The key question for Baty – and all university rankers – is how willing they are to take the risks of engagement with the sector, and the serious critique of their methodologies. Consensus may not be possible – academia is perhaps too fractious and ranking organisations are clearly constrained by their commercial interests – but think of the impact we might have if we could find a sustainable way to work towards a common goal.

 

Posted in Research Assessment, Science & Politics, Scientific Life | 1 Comment