It is a truth universally acknowledged that scientists who take greater risks are more likely to make important discoveries.<\/p>\n Actually, I\u2019m not sure it is a truth, and I don\u2019t know if it is universally acknowledged \u2013\u00a0I haven\u2019t looked closely enough at the evidence \u2013\u00a0but it is a long standing and widely held assumption. However, research funders struggle to operationalise this assumption and the question of why that is the case is the focus of a stimulating recent paper in PLoS Biology from Kevin Gross and Carl Bergstrom on \u201cRationalizing risk aversion in science: Why incentives to work hard clash with incentives to take risks<\/a>\u201d.<\/p>\n Gross and Bergstrom bring an economic perspective to the question that provides an insightful framing for some of the wider issues that it raises. Here\u2019s their abstract (with my emphasis and annotations):<\/p>\n Scientific research requires taking risks, as the most cautious approaches are unlikely to lead to the most rapid progress. Yet, much funded scientific research plays it safe and funding agencies bemoan the difficulty of attracting high-risk, high-return research projects. Why don\u2019t the incentives for scientific discovery adequately impel researchers toward such projects? Here, we adapt an economic contracting model to explore how the unobservability of risk and effort discourages risky research<\/strong>. The model considers a hidden-action problem<\/strong>, in which the scientific community must reward discoveries in a way that encourages effort and risk-taking while simultaneously protecting researchers\u2019 livelihoods against the vicissitudes of scientific chance. Its challenge when doing so is that incentives to motivate effort clash with incentives to motivate risk-taking, because a failed project may be evidence of a risky undertaking but could also be the result of simple sloth (I would add \u201cor incompetence\u201d<\/strong>). As a result, the incentives needed to encourage effort actively discourage risk-taking. Scientists respond by working on safe projects that generate evidence of effort but that don\u2019t move science forward as rapidly as riskier projects would. A social planner who prizes scientific productivity above researchers\u2019 well-being could remedy the problem by rewarding major discoveries richly enough to induce high-risk research, but in doing so would expose scientists to a degree of livelihood risk that ultimately leaves them worse off. Because the scientific community is approximately self-governing and constructs its own reward schedule, the incentives that researchers are willing to impose on themselves are inadequate to motivate the scientific risks that would best expedite scientific progress.<\/em><\/p>\n<\/blockquote>\n The authors\u2019 analysis relies on a mathematical model which I confess I did not completely understand*, so I\u2019ll spare you the details; (those who are more mathematically challenged will still get a lot from the paper if they confine themselves to the Introduction<\/a>, the box on Hidden-action models<\/a> and the Discussion<\/a>). \u00a0What I did understand of the model in terms of the codification of risk, reward, scientific value, effort, resources and utility and the explanations of simplifying assumptions seemed reasonable and, notwithstanding the obvious risks of confirmation bias, the two key conclusions that emerged from it resonated with my sense of how research decision-making works:<\/p>\n \u201c[\u2026] scientists seem either unable or unwilling to devise institutions that motivate investigators to embrace the scientific risks that would lead to the most rapid progress. Our analysis here suggests that this state of affairs can be explained at least in part by the interaction between two key structural elements in science: the unobservability of risk and effort<\/strong> on the one hand, and the self-organized nature of science<\/strong> on the other.\u201d\u00a0<\/em><\/p>\n<\/blockquote>\n The \u201cunobservability of risk and effort\u201d is more or less self-explanatory and accounts for the reliance on outputs such as publications as markers of achievement, which are to some degree beyond the control of the researcher. Demand for these outputs in the absence of an assessable record of the intelligence and invention brought to any research effort, is what leads many researchers to opt for safer, less-risky projects that are more likely to result in a paper, albeit one that reports a more incremental finding**.<\/p>\n The role of \u201cthe self-organized nature of science\u201d needs a little more unpacking for those who haven\u2019t read the whole paper. What Gross and Bergstrom mean here, I think, is that because the highly specialised nature of scientific endeavours relies so heavily on peer reviewers in the assessment of funding proposals, the key decision-makers have a strong internal sense of the risks attending project failure. They are therefore less willing than a hypothetical social planner charged with maximising scientific productivity to subject applicants to a reward regime that more punitively disfavours incrementalist approaches. The authors argue further that such hypothetical social planners cannot emerge in the first place because they would have to depend on researchers to determine how to value outcomes and would effectively morph into conduits for the collective view of the scientific community.<\/p>\n Several thoughts and questions occurred to me in the immediate wake of the paper\u2019s findings. They are not fully formed, so I offer them only in the interest of provoking further discussion.<\/p>\n Any researcher who has been on the receiving end of a paper or grant rejection \u2013\u00a0which is pretty much every researcher \u2013 would be forgiven for asking themselves how much more punishment they deserve at the hands of Gross and Bergstrom\u2019s social planner. Could it be that the current balance of risk and reward achieves the maximal level of scientific productivity that is commensurate with the desire to accord researchers a reasonable work-life balance? Personally, I don\u2019t think I ever achieved any kind of balance during my time as a jobbing academic trying to carve out a career in research. Current efforts<\/a> to incorporate research culture and the quality of the lived research process as part of assessment exercises spring from long-standing concerns about the risks and stresses imposed on researchers. One cost not discussed in any detail in Gross and Bergstrom\u2019s analysis is the human cost (though I appreciate they deliberately narrowed the parameters of their model to answer a specific technical question and the human cost factors into the appetite for risk).<\/p>\n Also, is it so difficult to uncover hidden effort? Better line management within research performing organisations could track effort, and perhaps even reward it directly if sufficient intramural funds were available. Researchers would also feel less of a sting from failed grant applications if funders were more open about the uncertainties in decision-making processes that ultimately rely on human judgement; feedback that clearly flags applications assessed as of fundable quality but for which funds were not available could provide some measure of career protection for researchers back at their home institution.<\/p>\n Although Gross and Bergstrom dismissed their hypothetical social planner, Daniel Sarewitz has argued powerfully and provocatively<\/a> for a more managerial approach to the organization of research, in part to tackle what he sees as the perverse inefficiencies arising because science is permitted undue freedoms to self-organize.<\/p>\n Elsewhere, Michael Nielsen and Kannjun Qiu\u2019s long but very worthwhile essay \u201cA Vision for Metascience<\/a>\u201d casts Gross and Bergstrom\u2019s social planners as risk-taking \u201cmetascience entrepreneurs\u201d empowered to achieve \u201cscalable change in the social process of science. Their ideas for incentivising risk include (among many other interesting but as yet untested suggestions) funding by variance, where grant applications are funded not by being high scoring among reviewers but by polarizing opinion; or failure audits, where grant programme managers are fired if the failure rate of their funded projects drops (yes, drops) below 50%.<\/p>\n Conceivably, the UK\u2019s recently established Advanced Research and Innovation Agency (ARIA<\/a>) also embodies an alternative form of the social planner. Modelled on similar Advanced Research Project Agencies in the US (e.g. DARPA<\/a>, ARPA-E<\/a>, ARPA-H<\/a>), ARIA is essentially run by programme directors in a range of topic areas who have the authority to select and fund projects they believe will result in the most significant breakthroughs. I\u2019m not aware of any study that has quantified the scientific productivity of the longer-established American agencies in comparison to more traditional mode of research funding, but their anecdotal (?) reputation was sufficient to persuade the UK government to bet on ARIA. I share the view that this a reasonable bet for the UK to take<\/a>, even if the funding agency has yet to develop robust criteria to demonstrate its own worth.<\/p>\n A last thought on productivity to throw into the mix: a hidden assumption of the Gross-Bergstrom model is that all research funding is awarded competitively. But what would be the impact on productivity of an ecosystem where researchers were provided with a basic or background level of funding, enough for a single postdoc or research technician, guaranteed for 10 years, in recognition of their hard work? As I\u2019ve argued elsewhere (and some time ago)<\/a>, such a regime could boost the productivity of the research funding ecosystem by reducing the wasted effort of submitting grant applications to funding systems\u00a0 that have chronically low success rates. Ghent University\u2019s introduction of a form of universal basic research funding<\/a> is a tentative, small-scale step in this direction.<\/p>\n There are further, broader questions raised by Gross and Bergstrom\u2019s paper. For example, is an analysis centred on individual economic actors weighing the balance of risk and reward in deciding which research projects to undertake the best way to explore questions of scientific productivity? What does it have to say about the impact of different institutional models, which not include not only the APRA\/ARIA approaches mentioned above but also experiments in Focused Research Organisations (FROs)<\/a>, innovative academic-industrial fusions such as Altos Labs<\/a>, or new types of research institution, such as Arcadia Science<\/a> or Astera<\/a>?<\/p>\n Finally, I\u2019m not sure how useful it is to talk about maximising or optimising the productivity of science, given the immense diversity and complexity of its processes, outputs and impacts. That\u2019s not to say that discussions of how to improve scientific productivity should figure out optimisation before any policy decisions can be made. Dare I suggest that more incrementalist approaches to improvement represent a more realistic and promising approach? We need to start somewhere \u2013\u00a0some already have! \u2013\u00a0and policy makers, no less than scientists, should be prepared to take risks. We will just have to work out the evaluation methods as we go.<\/p>\n I am grateful to James Wilsdon for a critical reading of a first draft of this blogpost.<\/em><\/p>\n \u00a0<\/strong><\/p>\n This is a repost of an article that was originally published on the Research on Research Institute website. Comments welcome!\u00a0 It is a truth universally acknowledged that scientists who take greater risks are more likely to make important discoveries. … Continue reading ![\"Black](\"https:\/\/live.staticflickr.com\/65535\/54021471603_01b6884144_z.jpg\")
<\/a><\/p>\n\n
\n
Footnotes<\/strong><\/h4>\n
*Although reasonably mathematically literate, I would have benefitted from a fuller description of the equations feeding into the model and suspect many other PLoS Biology readers would too. The Box on Hidden-action Models<\/a> was useful though!<\/h5>\n
**One of the simplifying assumptions of the model that did trouble me was that studies resulting in \u201cunpublishable outcomes\u201d, presumably null or negative results, were assigned a scientific value of zero. This is not altogether unreasonable given that null results are rarely written up for publication because they are not rewarded. Journals in search of citations to buttress their impact factors strongly prefer positive results and researchers in search of jobs, promotion and funding, are strongly incentivised to publish in \u2018top journals\u2019. But in truth null or negative results are not valueless and their absence from the published research record impacts productivity because the lessons learned from such studies are not logged, leading potentially to unnecessary duplication of effort. If we are interested in understanding the productivity of scientific research, we need better measures of the value of null results, not just a deeper understanding of the tensions between risks and incentives.<\/h5>\n","protected":false},"excerpt":{"rendered":"