Yesterday I had the privilege of participating in the 14th Berlin Debate on Science and Science Policy, which is hosted by the Robert Bosch Foundation. This year the invited participants discussed “Who owns science? Reshaping the Scientific Value Chain in the 21st Century“. There was a lot to unpack in that topic but for my formal contribution as chair of DORA (a six-minute opening statement), I took the opportunity to explore the links between open science, research evaluation and inclusion. My statement is reproduced below:
I have been asked to look at one of the key challenges on the road to Open Access and Open Science: How to secure and measure quality if we stop relying on the Impact Factor (and the peer review-systems) of prestigious, but paywalled journals.
The first part of that question, how do we secure quality, presupposes that the present system of scholarly publishing does a good job of quality control. Now I am not a fan of talk of a “reproducibility crisis” or of proclamations that “science is broken”; but clearly, despite the fact that many authors, reviewers, editors and journals are doing a good job, there are problems.
Our systems of research evaluation have become over-metricised. In particular, as we all know, we have an over-developed reliance on journal prestige – most often signified by the impact factor – as a measure of achievement.
This means that the dual functions of publication, first to share new discoveries and insights, and second to establish a reputation on which to build a career, have become unbalanced. The chase for a metricised reputation panders to selfish instincts and stokes hyper-competition by focusing on objects (papers and grants), rather than on people.
It has led to numerous perverse effects that undermine science:
- problems with the reliability of the scientific record;
- increased retractions, which undermine public trust;
- delayed publication which, if the public was aware of it, would further undermine trust;
- we have a very patchy record on data and reagent sharing;
- and a publication system that is still largely closed, not just to the public but to many academics.
Our obsession in research evaluation with metrics such as impact factors is an unhealthy breach of science’s social contract and increasingly puts scientists in conflict with the values that many claim underpin science.
Those values are rooted in the moral purpose of science, which is to make the world a better place – for everyone. They have emerged over centuries of continuous discourse. I don’t have time to excavate the history in detail so let me be content with a single footnote from Robert Merton’s famous 1942 paper on science and democracy, where he highlights remarks made by the 18th century French entomologist and thinker René Antoine Ferchault de Réaumur. Writing about the moral compulsion to publish one’s research, and while acknowledging national obligations, Réaumur ultimately concluded that “those who work to advance science and the arts should consider themselves citizens of the world.”
To me this is a 300-year old vision of open science. And it is my conviction that open science offers a route out of many of our present difficulties. It provides opportunities to reconnect with our most fundamental values, not just as scientists, but as human beings.
To be sure, there are some practical difficulties to be overcome in moving toward open science practices (inequities of cost, for example, that I hope we might explore in the next session), but there are no fundamental impediments and no serious threats to the quality of our research work. In fact, in very many ways, an open approach will improve our work because it demands that we expand our view of the qualities of research that matter and that should be evaluated.
The rise of preprints, for example, makes science faster; and because they present results independently of any journal brand, they provide a healthy focus on the quality of the content. Their openness (as with open access papers) means that they are available for the widest possible scrutiny – an incentive to authors to get things right – and for the widest possible re-use. Preprints also appear to foster commentary in ways that peer reviewed publications have struggled to do and this is helping to pave the way for experiments in open peer review (a practice that, if adopted universally, would kill off predatory open access journals).
Just as importantly, the rise of open science raises the question of why academic papers should be available to all. It is not just, as many governments reasonably contend, that the public has the right to see what they have paid for, and to demand maximal returns on its investments, but also because there are many stakeholders with a genuine interest: patient advocacy groups, environmental campaigners, citizen scientists – indeed any citizen concerned about the direction of travel of the modern world.
The dialogue engendered by open science raises the question of who should shape science. This is not just about engaging with the public, enabling them to have a say in setting the research agenda*. It is also, crucially, about taking a closer look at the fact that in no country that I can think of is the academy representative of the population at large. Women and ethnic minorities, among other groups, are chronically under-represented. This is in part a result of our present system of evaluation: metrics and rankings too easily sustain the status quo. How can an academy that is not truly global claim to be able to identify the world’s most pressing problems?
Open science compels us to re-examine what it is that we – the diverse citizens of the world – want from science, and to figure out how we are going to evaluate that. This is a project that DORA is keen to advance, through advocacy and speechifying, yes, but also by developing and promoting practical solutions.
To me open science represents hope: a chance to re-negotiate science’s social contract in ways that are more inclusive, more open to public dialogue on the proper balance between academic freedom and responsibility, and more rooted in the moral purpose that so many of us are seeking to give meaning to our lives.
