The issue of ‘impact’ appears here to stay in UK research. There has been much written about it, including by fellow OT blogger Stephen Curry here. With the draft guidelines for the REF about to be published, in which impact will also feature (in the form of case studies covering research already done whose impact is to be described in some detail, and ideally if impossibly quantified), it isn’t going to go away any time soon. My own comments on this blog have mainly dealt with impact in the form of those ‘pathways to impact’ statements required on all research council grant submissions. These, unlike the REF, deal with future aspirations rather than past accomplishments, a point neatly stated by Richard Jones in his own recent post on Impact as:
both HEFCE and RCUK want the idea of impact to have a greater influence on funding decisions. But HEFCE’s version of impact is backward looking and concerned with measurement, RCUK’s interest is forward looking and concerned with changing behaviours.
These latter RCUK statements have now attracted the ire of Mark Blamire writing recently for Physics World.
He complains
all parties – the funding agencies, scientists and publishers – are complicit in cloaking scientific research with a far greater significance than can be objectively justified. This is most obvious in the current focus on “energy” and the real danger of climate change. Governments have been persuaded to divert funding into energy research because it is a relatively inexpensive way of appearing to tackle the problem – at least in comparison with actually doing anything about it.
He goes on specifically to explore the actual outcome of making more efficient flat screen TV’s, to replace the old-fashioned CRT models. As he points out, it has become possible using the new technology to make much larger screens than ever before, and hence actually increase power consumption, far from saving it. Hence he concludes
So, if we are being really honest, an accurate summary of the “impact” of the physics-led development of display technologies is that it has enabled televisions to have a much larger power consumption – an outcome that is unlikely ever to have appeared in a funding impact statement.
Now, without doubting any of the facts he presents, I feel this is a perverse – if striking – way of looking at things. It seems to me a scientist who was trying to make more efficient devices of any type, would be justified in laying claim to that intent. That manufacturers satisfy the strange whims of customers to cover the entire length of their living rooms with flat screens so that they can follow their favourite soaps in larger than life forms, does not mean the scientist should not have set out to make more energy-efficient screens in the first place. Companies are always out to make money and will do so by any means available to them. Thus the only way round this is to alter what the consumer wants, so that the desire for energy-guzzling screens is no longer there and the market dries up. A tough challenge.
The combination of green forces, marketing mismanagement by some very large companies, and some interesting spin from the media managed to stop GM food production in its tracks within the UK (if not quite everywhere in the world), despite any – again perfectly warranted – impact claims that many plant scientists could have made about the ability of these crops to improve yields in a variety of ways. This would have been true, and even more advantageous, in many developing countries as well as the UK. GM crops are not simply about making money for companies like Monsanto, nor about destroying the Monarch butterfly; they have the potential for many positive benefits, particularly in parts of the world where production is most difficult, but ‘society’, loosely defined, has to a large extent rejected them. If the media chose to attack large screen TV’s, and the green party joined in their vilification, no doubt something could be done to cut energy consumption per household due to their use. But the fact that the research has been done to make them more efficient stands as a useful contribution in any case.
Of course Mark is right that many of these impact statements will make claims of dubious validity, and often the writer will be doing this with great cynicism, knowing that what is written is either over-the-top or stretching a point. That in itself may cause little damage, and I optimistically would still like to believe that the statements serve a useful purpose in encouraging a little thought about what the ultimate aims and outputs, broadly interpreted, of the work might be. (I also, as I have alluded to before, believe there is far more that could and should be included in these statements beyond what might loosely be termed ‘economic impact’.) I actually think it is the press releases associated with completion of the work that are far more misleading, in ways that quite often can be laid directly at the door of the university’s press team/communications office because they are always keen to maximise the effect of any story, in ways that may make the PI feel distinctly uncomfortable. Most PI’s will, possibly naively, think a little over-hyping may not matter; and in general it may not. Nevertheless, it is this effect that leads to the phenomenal number of ‘cures’ for cancer (or insert your alternative disease of choice here) university scientists apparently manage to find, which somehow never quite turn up at the local hospital.
However, let me return to the issue of climate change that provoked Mark Blamire’s diatribe against flat screen technology. It is not the impact statements that are or are not going to change the world, or even let the government off the hook of doing anything. What is needed is to change attitudes in ways that, very very briefly, it looked like the Stern Review might do. This week’s Guardian highlighted a different way a (US) climate scientist is setting about this, notably taking direct action.
[James} Hansen is the climate scientist’s climate scientist. He has testified about the issue in front of Congress, but has had enough of the standard government response – “greenwash”, he calls it. Last month, Hansen issued an uncompromising plea for Americans to involve themselves with civil unrest over climate change. “We want you to consider doing something hard – coming to Washington in the hottest and stickiest weeks of the summer and engaging in civil disobedience that will likely get you arrested,” he says in a letter on grist.org.
So maybe the corollary to Mark’s complaint is that our impact statements should read
I am going to spend each weekend in the second year of the grant participating in civil disobedience to draw attention to the dangers of the flat screen technology, and hence am likely to be in prison for the third year.
Or perhaps we need to do a better job of working in public engagement to encourage people not to want to buy gadgets with unnecessarily high energy consumption, and to indulge ourselves with more communication via skype (I don’t suppose anyone put that in their impact statement) to remove the need for gadding round the world. It is not the impact statements that will make a difference in this case, but other ways in which we choose to operate as scientists.
I think you are rather missing the essence of the point, which is rather elegantly illustrated by this example of flat-screen TVs. The issue is surely that the greatest true impact of any piece of research is likely to be entirely unpredictable: in this case, the unexpected consequence that flat screens encourage greater power consumption rather than less. There is therefore a signal-to-noise issue that if impact statements are taken at face value then, even if they are not amplified by hype, science will end up being driven by the noise of expected consequences, which for truly innovative research may well be drowned out by the signal of the unexpected.
Indeed, surely we should be giving preference to research that is sufficiently innovative that there is an expectation that the unexpected consequences will be far more significant than the expected ones, which makes reliance on the predictable consequences as a guide to selecting research areas even more damaging to the future of true innovation.
Hi Athene. You may know there was a Science Question Time devoted to impact at Imperial this week, with a podcast and summary available via CASE’s website. The panel were pretty positive about the shift to impact, if only to enable UK scientists to justify our worth to HMT.
A useful analogy mentioned by one of the panel members was the introduction of league tables in schools. Inherently a sensible thing, but one which encourages teachers to focus on pupils exam strategies rather than teaching per se. Similarly, I fear that the introduction of impact by HEFCE and RCUK may have similar unintended consequences. Well meaning, but an unheaval that runs the risk of suppressing research fields that at face value have little economic or societal benefit, yet might be central to a future not-yet-predictable advance?
HMT, on the basis of the strong track record of UK science, must have been sufficiently convinced of our collective merits to not cut the science budget (admittedly flat cash with severely reduced capital) last autumn. This track record predates impact in applications. Since only the best possible research stands a chance of succeeding in RCUK applications, the decider between two equally strong proposals may be one whose (future) pathways to impact statement overstates the likely benefit.
Finally, there is no talk of impact in German grant applications at present, so their funding agencies must still be comfortable with what their scientists produce without trying to measure it, perhaps because German industry expends a much higher fraction of its GDP on R&D than the UK? If UK plc wants more bang for its science buck, wouldn’t it be better to invest more heavily in, say, the Technology Strategy Board instead of adding additional hurdles to university research?
Michael, no I’m not missing the point. I would entirely concur that the unexpected is what makes research so interesting, and my post said nothing to the contrary. Research that only leads to foregone conclusions is hardly adventurous or going to advance the field, although sometimes it is necessary for corroboration (and sometimes, regrettably, it is the stuff that seems more likely to be funded). That was not where the emphasis of the original Blamire piece lay. On the contrary, it appeared to mix up different ideas. Firstly, the implication was almost that the research might have been better left undone because it led to an outcome of more energy being used – which I felt was something implicitly and unreasonably laid at the scientists’ door. Secondly, that somehow the scientists were being used as a ‘cloak’ as he put it, for the Government to do nothing much and, again by implication, that we were therefore complicit by writing impact statements. That is where the direct action comes in as a logical if improbable alternative course of action for scientists to take. Scientists are citizens as well as researchers, and we should take our responsibilities in the round seriously, although few would probably go as far as Hansen.
I think it is important not to lay too much blame and stress on impact statements, not to convolute all the ills of the current funding climate and societal issues with the existence of this much maligned requirement for a 2 page document. There are many positive things scientists can do. One of which is to think about plausible outcomes of our research. Getting these wrong is all too likely; but failing to think if there is anything that makes the work worth doing beyond idle curiosity at the taxpayers’ expense is not something that is going to be easily swallowed by the majority of the population. Nor should we expect it to be. Even blue skies research can be thought about and justified – as I have always said there is far more to impact than simple economics.
Paul, in my experience at BBSRC panels I don’t think it is the hype that convinces anyone about the ‘excellence’ of these statements. It is the thoughtfulness. I can’t judge other research councils and how they operate, but I have certainly heard it said by my colleagues that a good impact statement frequently ties in with a well-argued case, because the PI has sat down and thought things through in some detail. Most panel members have pretty good BS detectors in-built and can cut through the verbiage. But then, maybe I’m just an incurable optimist.
I am rather surprised by the attention that impact (from an RCUK perspective) has received. An impact statement in my view allows the proposer to articulate why they want to do what they state in the proposal. It allows scientists the opportunity to present in a thoughtful manner the bigger picture of their research. I don’t buy this belief that blue sky research suffers from impact statements. If the proposer really knows why they are doing this research then the impact statement should be easy to write.
I find much the HEFCE impact a little stranger. If the measurements of impact were purely to justify to the public why research is funded, I’d be fine with it. However, judging the current state of an institution by research that was done 15 years ago is something I just don’t understand.
The comparison with German research I think is unaffected by all of this, although it is a related matter. Governments in the UK (of both persuasions) have shown that the University sector is something they do not understand particularly well. If they did they would not have been surprised by everyone charging £9k for undergraduate teaching; a University degree is not like a tin of baked beans. German industry can approach and collaborate with universities as and when they wish. UK industry seems to be contracting out its research to universities. The easier UK governments have made it for industry and universities to work together, the more corporate research appears to have suffered. It would be interesting if the government gave itself another metric by which to measure the performance of their industrial policy: by how many has the number of corporate research labs increased or decreased during one election cycle? Of course, there are two sides to this argument. German industry might be great, but then our universities stand up very well to theirs.
PS Paul can I have a lift on Tuesday?