This week saw the delivery of a funeral wreath and coffin to the Houses of Parliament. Why? Because a new grouping called Science for the Future decided to declare to MP’s the ‘death’ of British science. Actually what they were really complaining about was the way the EPSRC has been acting recently which, they claim, is mismanaging science funding to the extent that they would rather see their budget cut than continue as things are. Not everyone agrees with their actions. I’d like to explain why I, for one, am opposed to what they did in the way that they did it. Much of the background can be found on Pursestringtheory blog (with an extensive comment stream) so, for readers who haven’t been following the story, more detail can be found there, as well as commentary from Mark Henderson here and a thoughtful retrospective here.
I am no apologist for the EPSRC. Indeed I am sure that they have upon repeated occasions found me to be a thorn in their side, so it is all the more surprising to me both to find myself reacting so angrily to Science for the Future’s actions, and to get encouraging tweets from senior managers at EPSRC on the basis of comments I have made. I once spent an entire formal dinner haranguing Dave Delpy – EPSRC’s CEO – for their policy towards the sort of interdisciplinary research I do. He was gracious about it, but I doubt I got him to change his mind one jot. Such dialogue – tending to argument – between me and many others in the EPSRC team about the problems I perceive, particularly for work at the physics-biology interface, has been ongoing and visible for many years. Dialogue has been both semi-private – as at the dinner – and (relatively) public, for instance in meetings at the IOP’s Standing Committee for Physics Professors. As I say, I am no apologist for EPSRC.
But to return to Tuesday’s actions. The EPSRC, let us remember, does not cover all of UK science, it is the Engineering and Physical Sciences Research Council. The biomedics, the astronomers and geneticists, the environmentalists and many others do not get their funding from the EPSRC in the main. So, for starters, Science for the Future is making an overstated claim when it says that the EPSRC is responsible for the death of ‘science’. Even if one agrees that the EPSRC is damaging those bits of science they fund, let’s not encourage excessive and exaggerated claims, thereby muddying the waters about funding for everyone. However, as far as I can judge, much of the group’s actions/words are in this same overly dramatic vein. Stalin seems to have got invoked at some point as a point of comparison, which is not only inaccurate but pretty offensive. Why is it necessary to use such hyperbole, a point other commenters have made (see Mark Henderson’s commentary in the Guardian, or Richard Jones’ comment on the PurseStringTheory blog)?
Around 100 people accompanied the funeral procession to Parliament, with apparently a diverse bunch of concerns ranging from disagreement with the recent Shaping Capability decisions (particularly as it affects synthetic chemistry), to the requirement that all applicants for grants consider their potential ‘impact’. This latter point is one that Philip Moriarty in particular takes grave exception to, and he has been waging war on it since its introduction (see here for previous discussion between him and me on this topic). In general I may feel that much of what EPSRC has done recently has been badly handled, and that more robust mechanisms for consultation with and challenge from their community could only be a good thing. I would like to be more convinced that their Council thoroughly think through and challenge all decisions before they are released onto the community. But I fervently believe that taking disagreements to the politicians in this histrionic way can only be harmful: for science, for scientists and hence for the future of the country – since, by implication, we all believe science is vital for our country’s economic and social well-being (as Science is Vital so abundantly and successfully made clear in the run up to the last CSR). I do not believe (to use a legal phrase) this is a ‘proportionate means to achieve a legitimate aim’, however laudable some of these aims may be. My objection does not equate to wanting to issue a gagging order about public debate; it is simply that using up an occasion of engaging politicians for a stunt rather than for a reasoned presentation of a broad swathe of issues backed by a significant fraction of the community seems, at the very least, to be unhelpful.
Politicians are not, by and large, scientifically expert (with the notable of my own MP and erstwhile academic colleague in Cambridge Julian Huppert). By and large they don’t think about science very much, and they are hardly likely to appreciate the nuances underlying this week’s demonstrations. Nuances such as the fact the ‘science’ they claim to represent only covers a section of all science, and that their organisation – with its turnout only 5% of the size of the Science is Vital demonstration in 2010 – is not demonstrably representative of the community they claim to stand for. As far as I can tell the group’s primary aim is to bring down Dave Delpy and the senior management at EPSRC, but perhaps there is something more subtle they are aspiring to. Has Science for the Future attempted to establish how many people agree with these aims? I’d like to see the evidence if so. I, for one, would have felt far more in tune with the organisation if they had simply been content to draw up and make a presentation of a petition to our politicians in place of the horse-drawn carriage, which would have had the added benefit of enabling everyone to know how many scientists wanted to sign up with Science for the Future and its goals. As Mark Henderson’s recently published book Geek Manifesto so strongly states, and as Adam Smith in his commentary in the Guardian says in agreement, scientists should stand up and make their voices count and not be afraid of policy and politicians. But, as anyone who has got engaged with trying to effect a change in policy (in government or elsewhere) knows, there are good and bad ways of setting out to achieve this.
Philip Moriarty has challenged me what I think can be done, if not advancing on Parliament. Specifically he has said
Is your response that, regardless of what flaws we see in the research councils, we should keep our heads well below the parapet and not ‘rock the boat’?
As far as I can tell what he means by this is that anything short of publicly drawing attention to his complaints by something like the stunt of a coffin at Parliament equates to doing nothing. I think the world is full of far more gradations than this seems to imply. He seems to think it makes the actions of me and those of my ilk who think other routes more constructive, ‘supine’ and ‘unethical’, to use his terms. It is not sufficient for those of us more pragmatic beings to talk to EPSRC officials both ‘behind closed doors’ and publicly at the many events the EPSRC have held, where their managers have tried to put across the justifications for their actions in the face of widespread anxiety and even anger from the community. Furthermore, leaders of the Royal Society and relevant professional bodies have written in no uncertain terms to the EPSRC expressing unease about some of the things that have been done – again actions very much in the public domain. These events have hardly amounted to individuals lurking beneath parapets or making sure the boat does not judder more than a little. I simply cannot understand why Philip Moriarty thinks that because people like myself believe tackling politicians at this stage, with as little across-the-board support as this week’s event mustered, is the same thing as being (again to quote remarks specifically addressed to me by him)
seemingly happy that the character of basic scientific research in UK universities can be eroded, as long as the cash keeps flowing
with the added corollary that he believes what the ESPRC are doing is equivalent to eroding the scientific method. It would seem that those of us who are unhappy with dramatic media-visible exploits are to be deemed as implicitly spineless and content to see public money ‘squandered’ simply because we disagree with the specific actions we saw this week.
My primary objection to the course of action this group chose to take is that by so doing they have weakened the position of all scientists in taking their legitimate and broad concerns about science funding to politicians. They have acted so as to spread confusion amongst the MPs who lack specialist knowledge about what the fundamental issues are, regarding who is unhappy and about what. There is a vast difference between specific concerns about some (not even all) actions of one research council, and holistic concerns about the whole way science is funded in this ‘time of austerity’. Nobody is suggesting Science for the Future should be muzzled, let alone that speaking out in public is inherently a bad thing, but tackling Parliament should be saved for the largest issues when there is a consensus of opinion to back up claims. Science is Vital demonstrated this last time, when their message was heard loud and clear by David Willetts and colleagues, and that – coupled with many other positive and constructive actions such as the timely publication of Royal Society’s Scientific Century report – led to a less than catastrophic CSR for science.
What worries me about the way the event and subsequent debate has emerged is that we are in danger of squabbling over the detail, however important, not concentrating on the big picture (and I do not intend on this blog to get drawn into the specifics of whether this or that area should be funded, or whether the whole idea of impact is equivalent to the devil incarnate ). I would urge Science for the Future to consider how representative they are of the whole of the EPSRC community, before they purport to act on our collective behalf by producing grandstanding exaggerated statements in order to gain publicity in the media. That so many of the signatories to the recent letter to the Telegraph, however illustrious they may be, live beyond the UK’s shores hardly demonstrates representation, nor confers confidence that all of them – just like the MPs – are really able to appreciate the nuances of the issues. I would like the organisation to think whether using up a rare chance to engage politicians on a relatively narrow – albeit important – issue is the best way of promoting the science they love so much. And to justify to people like me, who were not satisfied by their action, why they feel existing action groups with a well-established and trusted political pedigree such as CaSE (I’ll admit to a potential conflict of interest here, since I am a member of their Advisory Council), or a demonstrable effectiveness plus widespread grassroots support, such as Science is Vital, are insufficient to further the aims of putting the case for science funding and its appropriate management.
In the meantime they should think much harder about the damage that may be consequent on their actions by introducing unnecessary and confusing divisions in order to further a course of action that smacks of witch hunting against Dave Delpy personally, and which has seen unattractive invective directed at other individuals who have expressed unease and opposition to their actions. They need to recognize that disagreement in others does not imply that we are all lying on our backs doing nothing, happy to ignore misguided actions by looking the other way. It merely means not everyone will applaud stunts which are counterproductive or think that confrontation is helpful just because more civilised routes have failed. Maybe they should even understand that shouting long and hard about particular policies, long after fellow scientists have listened and turned away, whilst simultaneously accepting EPSRC funding in large swathes, isn’t a very coherent stance nor one likely to have productive outcomes. They remind me of British travellers overseas who think shouting louder in English will make locals understand them better – and they are about as likely to succeed.
Thank you for this post, Athene, with which I wholly agree. I confess I was shocked when I heard you’d been accused of being ‘supine’ – all I could think was, the person who said that obviously doesn’t know you very well!
Many scientists have cause to be grateful for your behind-the-scenes efforts on our behalf at the Royal Society and elsewhere.
Why do you believe this is counterproductive, exactly? Because it risks confusing MPs? Because it leads to divisive arguments such as that between Moriarty and yourself?
Frankly I’d suggest that “insufficient interest to read up” causes more MP confusion than imprecise sloganeering ever could. And from what you’ve quoted, it doesn’t sound like Moriarty is handling himself very well here, but him aside I’d still suggest that a rapprochement, a certain acceptance of diversity of tactics, is useful.
Indeed, I’d suggest a slightly different perspective on actions like this one — that they can complement more conventional advocacy work. I’d suggest that they’re advocating interests, not particular policy goals (to the extent that they have to articulate goals, it’s only because it’s hard to sound passionate and committed without any). Indeed, I’d suggest that the goal here is threefold: to physically confront parliament with evidence of discontent with their policies, and to make it harder for them to ignore it; to inform and motivate the public on this point (to clarify what austerity means for science, and how we should feel about that, even if you’re not exactly following EPSRC policy); and indeed to reinvigorate the participants themselves. Moriarty evidently isn’t satisfied with conventional advocacy. Would you rather he just gave up?
It’s only work like this that makes reform possible at all — it’s precisely the death of direct action movements in the UK and US that made reformers lose so much ground in recent memory. Discouraging them is the last thing we want to do.
I do not understand why you have been so upset by this over the past few days. Sure, Philip’s comments have not been the wisest. However, it was my impression that one of the main complaints about EPSRC, that more and more of their funding is ‘directed’ and given to individuals based on their perceived quality rather than peer review, is a valid one. I thought that it was sad to see that the most recent PST blog post was not able to explain these points properly nor provide proper evidence to back up such statements. Now it seems like all this was just a temper tantrum by some upset organic chemists. By the way, I think you too may have to be carefully with statements such as that they are not representing the majority of scientists. You do not know that this is the case and additionally I would be willing to believe that many would not want to speak out for fear of upsetting EPSRC.
Thanks for posting this, Athene. In relation to the exchange on PST and Philip Moriarty’s comments there, I would like to make one thing clear, speaking for myself here. It’s not that I secretly agree with SFTF’s position but think it’s being politically unwise to raise these issues publicly. On the contrary, I think many of their objections are wrong in principle or based on misconceptions about how science funding actually works now or has done in the past. I’m probably more sympathetic to EPSRC than Athene is (as we’ve discussed many times!); not that everything they do is always beyond question but I don’t think people always appreciate the constraints from government that they operate under and the limited time and resource they have as they try to do their best for UK science in the difficult circumstances we’re now in. I am worried – very worried, actually – about the future for UK science in the current economic times, but I think EPSRC is the wrong target for those who share those worries.
Mordecai
Philip and I can disagree – or agree – as much as we like and it in itself does not matter. It is counterproductive because confusing MPs with muddled goals and statements such as the ‘death of Bristish science’ are hardly going to facilitate improving anything. Yes it can draw attention to a situation SFTF’s followers are unhappy with, but in itself it won’t do much more (at least that is my belief). I have never suggested Philip shouldn’t express his discontent – as I say I’m not interested in gagging orders – but I am unhappy if it is done in this way, wasting a rare opportunity constructively to engage with politicians.
Anon
WIth so many scientists with no significant interest in EPSRC activities (being funded by other research councils), I cannot believe this group represents ‘the majority’ of scientists. Even if every EPSRC funded person were astonishingly of one mind, I don’t think that would amount to anything like a majority of scientists. It is also worth saying that at a grant-giving panel it is not ‘the EPSRC’ who make decisions, so upsetting them is unlikely to be very relevant. I suppose it is possible that there are situations where some sort of implicit blacklisting of individuals who have spoken out are excluded from some funding round, but I find it very hard to see how that can be. Despite some of the comments from SFTF’s supporters, criteria are transparent, even if unpalatable to some. Furthermore, as was pointed out in response to a comment about sour grapes – and I apologise I can’t remember on which blog this response came – the followers of SFTF are most certainly not without funds from EPSRC. So, it does not look to me like Philip, for instance, is suffering from speaking out (although I cannot be 100% sure that he has had new funds since he started speaking out quite so loudly about impact).
Richard and Jenny – thanks for the encouraging words. Richard is absolutely right, though, he and I have had innumerable heated debates about EPSRC with me being the more critical one.
Ah. But I think you’re underestimating how much good a pithy slogan and a sense of humor can do. Sometimes that’s what makes the difference between “inevitable” and “political suicide.”
For what it’s worth, from my experience as an activist, I don’t think you and Phillip are in competition here. (Was this really an opportunity to “engage productively,” wasted or no?) Before long, I think that stunts like this will make your job easier, and I hope you’ll be open to reconsidering this position if you see that happen.
Hi, Mordecai.
I agree. It would be nice to think that Athene and I (and, more broadly, Science For The Future) aren’t in competition. After all, we all want the same thing.
I’d also appreciate it if you could read over my response to Athene’s post below (or is it above? – depends on whether you’re reading this on a laptop/desktop or a mobile device like an iPhone!) and let me know which aspects of this you find unconvincing. (You say in your original post that I’m not handling myself very well and I’d genuinely like feedback on how to improve how I communicate the issues. Thanks.).
To address Athene’s comment re. my current funding situation (in her response to you) the current situation is this. Since April 2009, when EPSRC introduced the impact statement requirement, I no longer review (and therefore no longer submit) EPSRC proposals which require the “Pathways to Impact” statement. This stems from this call for a boycott of review of the Pathways to Impact statement. I resigned from the Peer Review College due to this boycott.
(Note, however, that my concerns are much broader and much more fundamental than red tape. Having spent a major fraction of the last few months preparing EU training network proposals (Marie Curie, Erasmus Mundus) – and coordinating multi-partner Marie Curie networks previously – I am more than familiar with ludicrously high bureaucratic overheads. For me, the much more important issue is the question of compromising the scientific method).
I currently hold an EPSRC Leadership Fellowship. I am very grateful to my peers for awarding this to me*. Recently, EPSRC announced a new tranche of funding for fellows who were awarded either a Leadership or Career Acceleration Fellowship in 2008 – 2010. This bypassed the standard application + expert peer review process and, importantly, did not require a “Pathways to Impact” statement. I therefore applied for the additional tranche of funding (which amounted to £240K on top of the original fellowship award of £1.5M) – something which still causes me sleepless nights given that I think EPSRC’s focus on leaders (as opposed to the best ideas) is wrong. With this additional funding, my EPSRC Fellowship runs until the end of March 2014. After which, I will not be applying for/reviewing EPSRC grants unless the “Pathways to Impact” requirement is removed.
* I find it just a little disconcerting that Athene and Richard (and, more importantly, EPSRC itself) discuss grant awards in terms of the Council awarding the funding. EPSRC shouldn’t be responsible for selecting who is awarded funding. It should be there simply to disburse the funding. But with the Council’s change from a funder to a sponsor of research, this is no longer true. This is the fundamental origin of many of SFTF’s criticisms of EPSRC.
Philip
P.S. To head off the “ivory tower academic” cliche that is often unfairly levelled at those deeply opposed to the “Pathways to Impact” requirement, and as Athene knows well, I am very committed to public engagement. (See, for example, Sixty Symbols ). As I said in a Physics World commentary a few years back (“The Economic Impact Fallacy”),
Hi Philip,
I was originally reacting to “supine” and to “unethical” — but now that I look up the original comment, I see that I completely misread you. (I thought it was a personal attack, but you were instead characterizing a proposed strategy?) So my mistake, and I apologize.
Hi, Mordecai.
Thank you for that. I am delighted that you feel that my “supine” comment comes across as a criticism of a strategy rather than a personal attack. I am very well aware that Athene has certainly not been shy in raising criticisms of EPSRC at various meetings – “supine” and “unethical” were certainly not targetted at her personally or, indeed, anyone else who has been critical of SftF.
Best wishes,
Philip
Hi, Athene.
Spent yesterday with my family. A day spent doing fun things with my three children (who are now three-, six-, and eight-years-old) helps put things into perspective. Also thought that, for a change, instead of dashing off a response last night I’d get a good night’s sleep in before replying.
You make some fairly strong charges against those of us who deign it worthwhile to criticise EPSRC in public. As Anon points out above, however, you should be careful of assuming that dissatisfaction with EPSRC is not widely held and that your position is the majority view. Where is your evidence?
Perhaps have a chat with some of the younger members of staff in your department and ask them whether they think it’s appropriate for EPSRC to award “Dream” fellowships on the basis of selections made by EPSRC programme managers; to limit fellowships to certain ‘strategic’ areas selected by EPSRC; to blacklist researchers on the basis of an entirely unfair – and, yes, I’ll say it again – unethical process; or to distort the scientific method so that impact “informs the direction of your research”.
(On this latter point, and as I’ve noted over at the Purse String Theory blog , I don’t think it’s an exaggeration at all – and certainly not ‘risible hyperbole’ – to suggest that the focus on impact compromises/distorts/perverts the scientific method. After all, we don’t teach undergraduates to design experiments to give them the results they “expect”… Moreover, the impact policy is RCUK-wide , not just limited to EPSRC. As you know well.).
In any case, that some EPSRC fellowships have been given the cringeworthy label “Dream” is, by itself, more than enough evidence that there’s something badly wrong at the heart of the Council! Who the hell was responsible for dreaming up the “dream fellowships” monicker? First against the wall when the revolution comes…
(Just to be clear. That was a joke. Promise. I don’t actually mean that I want to take EPSRC staff out and have them shot. Really, I don’t. Indeed, I actually rather like Atti Emecz, EPSRC’s Director of Communications. Anybody who is a NoMeansNo and a Quicksand fan can’t be all bad!)
As I also mentioned in that e-mail to you and others last week, Ananyo Bhattacharya, Nature’s Chief Online editor, perhaps put it best – and certainly most pithily – when he tweeted:
In any case, and without you apparently realising, you make the case for public criticism of the research councils in your post. You state:
“…but I doubt I got him to change his mind one jot” . Precisely. And that has been the situation throughout discussions/exchanges with EPSRC over the last three to four years.
Town Meetings come and go.
‘Consultations’ come and go. (EPSRC has, of course, redefined the meaning of the term ‘consult’. Apparently the OED isn’t good enough for them).
Discussions under the rather troublesome Chatham House Rule (this is public money, after all!) come and go.
Criticisms in international reviews – including, in the last International Review of UK Chemistry, a call for a panel of internationally leading scientists to review EPSRC policies and processes – are ignored. (ESPRC’s reponse to the suggestion that it be subject to an international review? “No action required”).
Delpy and Armitt are dragged up in front of the Science and Technology Select Committee and roundly criticised, delivering what was frankly a farcical performance at times. (Note that the Chair of the S&T committee used the term “sophistry” to describe ESPRC’s interactions with the researchers it funds.)
And yet nothing changes.
You explicitly acknowledge this – your “haranguing” of Delpy led absolutely nowhere. Welcome to the club! (Perhaps you might like to sign up to Science For The Future? It’s so frustrating when valid arguments fall on deaf ears, isn’t it? And for what it’s worth, I think EPSRC does a pretty good job of encouraging interdisciplinary research. Much of the criticism I hear is about the overall quality of reviewing of interdisciplinary proposals. But this is outside EPSRC’s control – even I admit that the Council can’t be held accountable for everything!)
The only time that EPSRC has changed its policies in recent years was when a very public petition led them to rethink their underhand original approach to the blacklisting scheme (which was to be introduced retrospectively).
The only thing that makes a difference is (very) public pressure.
As Mordecai points out above, what better way to do this than via a pithy slogan and attention-grabbing campaign? And if the scientific community’s arguments for continued funding for science are fatally undermined by SFTF’s criticisms of EPSRC then we are in a very bad place indeed.
Philip
P.S. You say in the e-mail you sent me to alert me to your post above that “In the spirit of open debate…I am not interested or willing to engage in debate [here] about the specifics of what you disagree with in the EPSRC’s actions and policies” . Fine. There’s much more about the specific difficulties with EPSRC at, for example, Sheer Lunacy and at Girl, Interrupting .
I was not referring to the, apparently 60%, that goes through the responsive mode panels but the remainder that goes to directed programmes, platform grants, programme grants, etc. where there is more scope for EPSRC to influence the outcome. What scares some people is the suggestion made by EPSRC last summer that they would move around the rank list arrived at by the panel based on peer review post panel. It may be that this is all done in good faith and under the pressure of tight budgets but it gives the impression that something inappropriate is going on.
The 60% figure for responsive mode was one of EPSRC’s “myth-buster” tweets, if I remember correctly?
The 60% claim spectacularly misses the point. Responsive Mode has been entirely redefined due to Shaping Capability, National Importance, and, of course, Impact. EPSRC cannot claim that it is supporting exploratory/basic research and at the same time ask that impact informs the direction of your research (see, ad nauseum, my comments at String Purse Theory ). It’s a dictionary definition of an oxymoron!
Is there testimony from anyone participating in EPSRC’s peer review panels that the impact statements are having the corrosive effect that you claim?
I had this fear of the BBSRC’s practices but was relieved to see when I joined one of their panels that scientific excellence was first and foremost the criteria for ranking grants. Impact was only ever used as a tie-breaker between two grants otherwise considered equally good. This secondary use of impact statements was recently publicly endorsed by the BBSRC Chief Exec. Doug Kell. If the practice of peer review panels at EPSRC was significantly different from this that would be cause for concern — but it would be good to hear one way or the other from panel members.
A few more points occur to me.
Firstly, on the question of whether it is right to criticise scientists for coming out in public with their concerns about research councils. To my knowledge, Philip Moriarty has been criticising EPSRC publicly for at least 5 years, and I’ve never before criticised him for that either in public or in private. He has a very principled position based on pure science values and the primacy of peer review, and it’s to his credit that his position is entirely unconnected with whether or not he personally has been able to find funding for his research. I don’t entirely agree with it, and I’ve been very happy to discuss why in articles like this one. What’s different now? Swapping arguments in the pages of Nature Nanotechnology is quite different to hiring a PR firm to lobby parliament, for all the reasons Athene gives in her post.
On impact, Philip says “we don’t teach undergraduates to design experiments to give them the results they “expect””. No, but we do teach undergraduates to understand why they are doing the experiment. The question of why you do an experiment is different to the question of what results your experiment produces; “impact” is to do with the former, not the latter. It’s about the context that frames your research; scientists are already adept at claiming that their research will lead to all sorts of benefits, and the idea behind “pathways to impact” is to help scientists think through how these benefits might in practise be realised. I don’t think this is at all without pitfalls – there is a real danger that this could exacerbate an already existing cycle of overpromising – but I think it’s wrong to say it fundamentally corrupts the scientific method.
On EPSRC’s non-response to consultation, what I wonder is how many of the current objectors have provided evidence to support the representations they make. Everyone has an opinion, as the saying goes, and opinions aren’t very helpful to EPSRC or anyone else unless they are supported by evidence. My own experience of being consulted as an IOP nominee for the shaping capability exercise was that, having spent some time thinking about what evidence would be helpful, doing a bit of reading to track some of it down, and making the effort to marshall it in a clear way, my arguments were listened to and I see them reflected to some extent in the outcome.
No, sorry, Richard. What you and Athene believe EPSRC/RCUK says about impact and what it actually says (and requests) are two very different things.
I point you, yet again, to RCUK’s #1 tip on how to write a good Je-S (i.e. grant) application:
Just so we’re ‘on the same page’ (ahem), let’s drag out the OED to check the definition of “inform”:
“inform, verb…. 2 [with object] give an essential or formative principle or quality to ”
Now, unless RCUK has decided also to redefine the meaning of the verb inform (along with “consult”), this is very different to your interpretation of what impact is all about. It is, however, entirely consistent with the claims throughout RCUK documents (and claims backed up publicly by Delpy et al.) that what it wants to do is embed a “culture change” in UK academia.
This is what pisses me off so much about impact – exploratory scientific research is no longer ‘good enough’. A scientist must be driven not by a quest for knowledge alone but by anticipating the impact of the work and, as described in that ‘tip’, adjusting their research accordingly.
That this is so ‘out of kilter’ with my motivations for becoming an academic scientist might explain why I spend so much of my time debating the matter.
Thank you , however, for the very kind words about my ‘stand’ on this. As you know, although we disagree on a number of aspects of science funding policy, I have a huge amount of respect for your well-considered and authoritative articles and blog posts on the topic.
Philip
…and as regards teaching the [undergrad] students “why they do the experiment”, I have never told an undergraduate or graduate student here at Nottingham (or elsewhere) that the reason they do an experiment is to produce an application which will have direct socioeconomic impact or – importantly – even indirect socioeconomic impact in the far-flung future.
They do the experiment to interrogate Nature and to learn a little more about how the universe behaves. Free of near-market considerations. Free of commercial constraints. Free of political interference. Free of preconceptions.
Are you, Athene, and James, like EPSRC, saying that this is not enough? (And let’s lay aside ethical issues for now – I’m a physicist working with matter under ultrahigh vacuum at temperatures of ~ 5 K).
Philip
“In any case, that some EPSRC fellowships have been given the cringeworthy label “Dream” is, by itself, more than enough evidence that there’s something badly wrong at the heart of the Council! ”
Yeah, it’s always sad when organizations jump the shark.
Philip, I spent Thursday and Friday doing a review of Lancaster’s physics department, where they still had up the posters for your seminar on Wednesday there – “Mechanical Atom Manipulation: Towards a Matter Compiler?”. You frame your very exciting research not in terms of a quest for pure knowledge, but by reference to an instrumental goal – “a matter compiler” – what else is this than anticipating the impact of your work?
Oh, come on Richard! You know that the matter compiler thing is a nice tag on which to hang the research. Do you really think that I am aiming to develop a Drexlerian nanofactory on the time-scale of my fellowship, or even my career?
What I’m interested in is the ultimate control of matter. Can we, for example, do ‘deterministic epitaxy’ where we control the trajectories of all atoms which ultimately comprise a nanostructure.
The fundamental physical chemistry/chemical physics questions this raises are what drives me, not the idea that one day I’ll roll out a nanofactory.
I’m just as capable of ‘spin’ as the next nanoscientist…
Philip
Philip, I think Richard makes a very good point. If you are using your impact statement in your seminar titles, maybe your research isn’t as pure and “blue skies” as you might think?! (I’m teasing you here!).
I don’t interpret impact as “I will make a nanofactory”, rather it’s the work I’m doing (if it goes well) could make a significant contribution to the physical principles that might be used in future nanotechnology engineering. Impact, in my opinion, is an exercise in separating “why I want to do this work” from “why should someone give me money to do this work”. I guess the goal is that basic researchers will think more about how to communicate outcomes to the relevant stakeholders so to increase the chances that someone will take these ideas to the next level of application rather than merely getting lost and forgotten in the voluminous academic literature.
Paul,
The key thing is what actually motivates my research. I am interested in understanding the fundamental limits in our control of matter – not whether I can exploit this understanding to develop a new technology + secure the IPR + lock down exclusive licences + commercially exploit it.
Athene has castigated other scientists (elsewhere at her blog) for writing an impact statement exactly along the lines you suggest (i.e. with nebulous statements about the long-term commercial impact. EPSRC itself advises against this).
If we in Nottingham develop the knowledge required to construct a “nanofactory” along the way then fine. But that does not mean that I am motivated by the development of a technology.
In any case, I publish my work in the open literature – just as a publicly-funded academic scientist should. But this immediately can lead to problems in terms of commercial exploitation. See, for example, “Science Mart” by Philip Mirowski, published last year by Harvard University Press. (Note that Mirowski calls into question much of the ‘standard/received’ wisdom about the public good character of academic research so it’s an uncomfortable read for me. It is, however, an authoritative appraisal of the damage that a strong focus on near-market impact can do to scientific progress).
David Willetts, to his immense credit, recognised the key economic impact of academic research when he pointed to the “absorptive capacity” of the country. But that was before he was brought back swiftly ‘on-message’ to sing the virtues of near-market impact.
Philip
The effect of “impact” on the quality of research is very obvious in my area (at the synthetic biology, physics, engineering interface).
Here there are two very obviously different approaches to research: the engineering approach involving getting to the specific goal as quickly as possible, and the underpinning science approach involving maybe not getting anywhere very fast but trying to understand the fundamental principles of the systems.
In the short term the engineering approach “wins”. But at the end of each engineering project you have to start again from scratch with your new goal in mind…or in other words very little that is transferable is gained.
In the short term the underpinning science approach “loses”. But you are learning things that apply generally, to all goals, and in the end you have to understand the fundamentals in order to be faster at achieving goals in the future. So in the long term it “wins”.
Another way of putting this is that it is quicker to just make something work than to actually have understood how and why it works in a way that is applicable to the next project.
The increasing emphasis on impact and short term dividends is making it very difficult to get funding to actually understand the things we have in some cases already created. But without that knowledge we are condemned to the same cycle of starting from scratch with each new goal.
I think it is a terrible shame that the EPSRC cannot lift itself above this mediocrity, especially as there are so very many examples of cases where fundamental understanding has proven critical in developing new money spinning applications slightly further down the line.
As far as I can see it has never been demonstrated that short termism or entrepreneurship is a more effective way of obtaining technology and societal benefit from research. than sponsoring fundamental research without any obvious immediate pay off.
@BB.
Thanks for an extremely important and perceptive comment. Note that John Armitt, until recently the Chair of EPSRC, stated that the purpose of EPSRC is “… to make sure that we are supporting those things that industry says it needs but which industry itself is not willing to fund.”
This quote alone explains just why EPSRC has lost its way so badly.
See Mike Duff’s excellent article in The Guardian from March for more – Research Council sacrifices basic science on the altar of commerce .
There are also a number of articles listed here which discuss the economic rationale for state support of basic science.
Peter Coles (‘Telescoper’) has also written an important blog post on last Tuesday’s events from the perspective of an STFC-funded researcher: EPSRC Blues .
Best wishes,
Philip
I would love it if my research had impact, and in fact gain a certain amount of satisfaction if it does (sorry to be so positive). I gain satisfaction from someone reading my paper and citing it, to it being discussed in a textbook, and working closely with a company to resolve a problem, using my “expertise”, are just three examples of where impact has made me think I do a worthwhile job. Forgive me for indulging there, but I have to put my small achievements in context.
On the other end of the scale I am immensely impressed by people whose research makes a very real impact to people’s lives. I went to a talk not so long ago by Robert Langer (famous tissue engineering pioneer at MIT). It was one of those talks where the speaker reminisces about their achievements. These achievements were not the papers cited (far more than most of us could hope to achieve by the way for this particular individual), or the books written about his work, but pictures of patients that his research had helped. Now that might come across all worthy, but it was inspiring enough for me to think beyond the realms of my research to where it could have real impact. Is that not what the Impact statement is about – challenging us to think about what could be? Surely we should be thinking like this when potentially receiving tax-payers money. Of course it is not always clear that our research can have immediate impact in the way that Langer demonstrated, but nevertheless I think it is a worthwhile excercise to map out where our research sits in society, which leads me on to my next point.
The EPSRC is in a difficult position at the moment – yes, it is! There are a number of critical issues that that government is, and should be, concerned about. Health of an ageing population, energy…. I needn’t go on here, you get my drift, and in any case you will have read the wording of all their documents on this matter. These are really critical issues, and in the past the government would have had national laboratories that could deal with some of these. Unfortunately, due to previous administration these have now pretty much dissapeared. We also realise now that the UK needs to get its manufacturing industry back on its feet, and indeed manufacturing is coming back. I was at a University in South Wales the other week and Tata Steel are investing £800M into manufacturing. There is even talk of opening up old coal mines again to feed this industry. This needs a scientific investment as well. We do need a scientific infrastructure right now within UK Universities that makes a real go at tackling these problems, otherwise we are in serious trouble.
Large scale investments in technology have in the past come at times of war, and in some respects we should have a war-like mentality with some of these issues. After all if predictions are correct there is very little time, and so tackling issues of energy and water resources (hence why this is, for example, highlighted for fellowship applications) ought to be a matter of great urgency. Manufacturing is another critical issue, as this is increasingly heading out east, and the UK will begin to seriously fall behind. Surely these issues ought to help us sharpen our pencils on impact? Actually, I find it quite an exciting time to be part of the scientific community. Rather than railing against the authorities that manage funding in this country we should be asking the question “How can we help?”. If our particular research theme that we have spent our careers on does not fit within this framework then maybe it is time to evoke one of the sayings of Ernest Rutherford: “We haven’t got the money, so we’ve got to think!”. Right, back to that EPSRC proposal…..
Hi, Steve.
There’s also a quote from Rutherford’s research supervisor (JJ Thomson) that’s rather apposite here:
Impact erodes the exploratory character that should be at the heart of basic science.
Of course a national funding council should support work spanning the spectrum of research from engineering through applied R&D to fundamental science. I’m sure we can agree on that. For engineering and applied R&D then of course consideration of impact makes a great deal of sense.
But EPSRC is determined that all research, regardless of where it falls on that spectrum, should be subject to a consideration of impact.
The arguments supporting impact are compelling at first glance – as you (and EPSRC) say, who wouldn’t want their work to have impact? But it’s much, much more complex than this. (And in any case, I, like very many scientists, do not decide which research problems to tackle on the basis of how much impact I predict they’ll have).
Robert Merton, the sociologist who first put forward a set of norms for science, had ‘disinterestedness’ as key among them. John Ziman, the physicist-turned-sociologist, argued that,
The Lancet, back in 2004, made the following extremely strong statement:
Why do you think that Merton and Ziman placed such high stock in the disinterested character of basic research? Why might The Lancet have made a statement as strong as that above?
Furthermore, in a paper commissioned by the Treasury, Salter and Martin had the following to say:
There is also increasing evidence to show that a focus on impact can actually slow down knowledge transfer to industry and impede innovation. I can give you chapter and verse on the literature to support this but one of the more intriguing (and heavily cited) papers on the relationship between target-driven science and innovation was published by Stern [Management Sci. 50 835 (2004)]. I find its conclusions fascinating:
[Richard (Jones) — if you’re still reading (!), I’d value your thoughts on Stern’s work and conclusions].
EPSRC’s simple-minded approach to encouraging a “shortening of the innovation chain” via impact – and make no mistake, this is what impact is all about – not only damages the fundamental disinterested ethos of basic science but has the potential to impede, rather than enhance, innovation.
Best wishes,
Philip
Steve
I think your description of where you find impact in the work you do is excellent and encouraging. I don’t spot anything that says you aren’t doing the science you want, simply that in doing it you are willing and indeed happy to think about impact, wherever it may occur.
BB
I think you misunderstand that impact doesn’t have to be impact tomorrow. It isn’t – whatever Philip may believe and try to convince others of – about short term impact. And, to use a legal analogy again, I think all the precedent I have seen on panels (admittedly more of BBSRC than EPSRC ones, but this is supposed to be an RCUK policy) suggests that a strong impact statement is one which shows reflection of what might going to be impact on whatever timescale is appropriate, rather than one that shows a short-term impact in a narrow and perhaps unthougtful way. So case-law says that Philip’s extreme view of his reading of statements about impact is not what is happening in practice. I discussed this in my earlier post which was prompted by reading so many Pathways to Impact statements.
Philip
I think everyone understands you hate everything about impact by now. As I said above, and in my email to you, I don’t want this post to turn into a detailed discussion of the minutiate of EPSRC’s policy, the pros and cons. I would therefore be grateful if you would take your valuable lexicon of quotes elsewhere to continue this particular debate. If you have further (brief) thoughts directly relating to the content of my post or about the wisdom of SFTF’s action, I am of course delighted to see them here. And I do agree with Richard and Paul, about your use of the phrase ‘matter compiler’.
Wow. What an erudite rebuttal, Athene! So, when you started your e-mail to me with the words “In the spirit of open debate…” what, precisely, did you mean?
I was responding to Steve’s comment, by the way. Not your post.
I, and many others, have been roundly criticised for misrepresenting EPSRC’s position. Please don’t misrepresent mine. Before you fire off a response in future, it’s best to take a little time to read the substance of what’s been said. (I speak from experience…). Note the following in my response to Steve:
In what sense does this suggest that I “hate everything about impact”.
(But, be honest, you didn’t get that far into reading my comment, did you?!)
(Im)politely telling me to get lost is hardly the way to foster constructive dialogue, is it?!
Philip
…and on the subject of constructive dialogue, I had a very interesting and helpful hour-long telephone conversation with Atti Emecz (EPSRC’s Director of Communications, as you know) yesterday. Atti doesn’t seem to be quite that disturbed by my use of quotes, results, and statistics from the literature.
Philip
Athene – yes I was just considering clarifying that I meant that while impact must be considered it is the focus on ‘immediate impact’ that is wreaking havoc in synthetic biology. It is also possible that it is the high impact journals that are causing more of a problem than funding agencies as it is IMPOSSIBLE to get and “ahh now we get it – it works like this” paper in to as high a profile journal as the initial “hey look what we did” paper.
The other problem is that even if you are correct, and long term potential is as valued as the short term potential, there is still a problem in that the predictors of long term potential do not appear to exist. I can tell you that understanding how proteins map their self-assembly energy landscape will one day be critical to nanotechnology but how do you know that I am right?
A lot of the biggest innovations appear to have come in areas that were not initially predicted to be important at the time the science was done. The science found its ‘big’ application somewhere other than where the researchers were aiming.
So why ask us to justify where our science might lead in the long term when there are so many examples of science changing society in such vastly unpredictable ways?
Is it not more honest to say “I am trying to understand the way nature works….this is bound to be useful in the end but I don’t know yet how that will manifest itself”?
Philip
I was not meaning to be impolite. I just feel the comment stream has headed off, as I feared it would, into discussions about EPSRC actions/policy which explicitly was not what my post was about. I was trying to steer discusson back to the central issue of the post and suggesting that a dialogue about impact was better had elsewhere. I assure you I always read all of all of the comments on my blog, but it is probably true my statement about you was too general. Nevertheless, your comments do indeed smack of having no time for anything to do with impact; whether that is an accurate description or not it is rare for you to have a good word to say for it. The balance of your comments is indubitably negative.
BB It is interesting that you are now saying the problem with Synthetic Biology may lie with the journals not the RCs. And, as I said in response to Philip elsewhere (probably on PST’s blog), I have seen – in a physics panel – the top awarded grant be pure fundamental (/blue skies) research but with an excellent description of long term impact, plus a discussion of the beauty of taking the work to science festivals and schools. I would have thought that should apply to the kinds of work you describe too.
Now I too am going to stop talking about impact (unless sorely provoked by further comments!)
<I was not meaning to be impolite.
That’s fine, Athene – no offence taken. (Well, not much). Sometimes we all say things in the heat of the moment that don’t quite have the subtlety and nuance we’d have liked (…ahem).
I’m intrigued that you feel it’s necessary for you to manage/direct the comment stream here! Impact is one of the most divisive aspects of EPSRC and REF policy and is one of the core complaints of the SFTF group. Your post was about the STFT event last Tuesday. It seems strange that you would want to quash/curtail debate on something that is one of the fundamental issues.
Steve and BB make some great points. Why can’t those be explored here?
Philip
I suspect stunts like SFTF’s coffin achieve exactly one thing: they allow those in Westminster and Whitehall to label the perpetrators as crackpots it would be futile to engage with. That’s perhaps harmless enough in itself, but it gets dangerous as soon as it morphs into the idea that all scientists are crackpots.
Interestingly, Owen, in the last few minutes I received a collation of the outcomes of the meetings that those who attended the event on Tuesday had with their MPs.
It is absolutely clear that the “perpetrators” are very far from being labelled as crackpots.
SFTF has been criticised for a perceived – and it is entirely perceived – lack of evidence for its claims about EPSRC. Please don’t descend to the depths of suggesting that politicians will label us as “crackpots” without having some very strong evidence to back up your claim.
Thanks.
Philip
And, by the way, at least have the intellectual courage to put your full name to your “crackpots” jibe. Otherwise, you’re just a common-or-garden troll…
Best wishes,
Philip (click on link above for full contact details and information)
Oh dear. That’s embarrassing. Somebody pointed out to me by e-mail that the link at the top of my previous comment doesn’t work.
For Owen, and anyone else who might be interested, the actual link is here .
Apologies.
Philip
Athene, I have heard about grants being slated on the basis that they seek to understand something that “has already been done” when already done is determined by someone publishing a one off “hey look what we did” paper with no attempt to understand the underpinning science. In this way the journals are influencing funding decisions by publishing the sexy and new as if it has the same or, god forbid, even more importance than boring “it took us 10 years to get to the bottom of this”.
Maybe this is an image/perception problem. But I get told day in day out that the impact is the most important bit of the grant, that without a good pathway document there is no point submitting. You counter to this is refreshing to hear.
Meanwhile on every paper I review, after 5 questions pertaining to how sexy and exciting I think the research is, there is a single tick box saying “are the conclusions justified by the data?” I have never yet read a paper that scores well on the first 5 questions and actually gets the tick. I think you can guess which are the more important criteria to the paper getting published….
Athene, I think your blog comments should have some “Je-s like” rules for Philip – 4000 character limits and no resubmissions! ;o)
Indeed, Paul. You see now why I don’t have a Twitter account!
Best wishes,
Philip
Let’s be even-handed – JeS like rules for everyone, not just Philip!
BB
If people cannot make clear what they are doing is different from what has been already published, they aren’t writing as well as they could. It should be possible to say Prof X has done such and such at a phenomenonological level, and now we are going to probe why this works – or something similar and appropriate. I think it is very important that myths are not propagated based on misconceptions or a tortuous path of Chinese Whispers.
Well in at least two cases I have seen the comments directly. People have understood that the big slash stuff is not sound from the proposal…but have rejected the grant for “being incremental” (isn’t all science of any worth built on the previous understanding?) and for “only seeking to understand what has already been done”. Not a goal I think one should be ashamed of.
When these sort of comments come from panels I think there is a problem.
Undoubtedly these may not have been the whole reason for the grant being rejected but I don’t believe they should have been even a small part at all.
Not my grant btw. 🙂
big splash….big slash is something else…
Oh sorry I missed the bit where you said you were not commenting on impact!
So regarding stunts with coffins….I have no idea if this action will increase or decrease public, government, EPSRC or scientists engagement with the issues under discussion.
In much the same way I have no idea if my blue skies research will one day be the corner stone of nano-tech, the corner stone of something I haven’t considered…or just another footnote in scientific endeavour. No one knows this about their blue skies research, not if they are being honest.
We are scientists – what we need are data.
SFTF need to demonstrate that their approach is more effective than the reasoned discussion amongst peers that is already ongoing.
EPSRC need to demonstrate that their current grant allocation mechanism actually delivers better value for money in both the short term and long term than a random selection process.
EPSRC also need to demonstrate that the impact statements improve the value for money over the previous impact free proposal system.
Right, 4000 characters and counting… (Damn just used up about 30).
We are scientists. We indeed need data.
But the most disconcerting thing with science funding policy is that, regardless of claims of the need for an ‘evidence base’, and like so much of politics, ideology will happily override evidence and statistics…
See, I can do it if I try!
Philip
It’s good to see that Athene’s post has prompted a helpful, informative and (largely!) good-natured debate about the pros, cons and future direction of the ‘impact agenda’. As a researcher working in the field of science policy, I very much welcome this: these are important issues which merit serious, evidence-based discussion. Philip mentions the work of my colleague Ben Martin on defining and measuring impact. He and several others here at SPRU are actively engaged in ongoing work in this area. I’ve also read, enjoyed and been provoked by Philip Mirowski’s work; indeed I’m looking forward to participating in an event at Lancaster in October on ‘The Changing Political Economy of Research & Innovation’, where Mirowski will be the keynote speaker (see http://bit.ly/zHcSo2).
But Athene does also have a point when she observes that the discussion has drifted some way from the focus of her original post. It’s possible to be critical and actively engaged in debates over impact (within EPSRC and elsewhere) without endorsing the particular tone and tactics of last week’s SFTF protest.
I’ve commented on this elsewhere and don’t intend to rehearse the arguments another time (Athene sets them out very eloquently in her post). Let me instead quote some extracts from a separate email exchange I had with Philip over the weekend in an effort to suggest some constructive (and less divisive) ways forward:
Like Athene, I am in no sense saying that scientists should remain silent and not raise concerns and questions about the policies of EPSRC, or those of other Research Councils and of government more widely. As I’ve repeatedly said, I don’t defend every action that the EPSRC has taken. But I do think that the tone and tenor of SFTT’s campaign against the EPSRC is way out of kilter with the scale of the actual problems.
Ths move that triggered such a sharp reaction from me, and many others, was to go beyond your previously narrow campaign about EPSRC towards broader claims about the ‘death of British science’. And I’d be genuinely interested a week on to know whether you consider last week’s action, and the reaction it has provoked, a success?
That we are having this debate, that there has been so much criticism of SFTF’s actions from across the scientific community, suggests to me that more work is required to build a convincing and evidence-based case of the scale and extent of the problems you have highlighted.
If I were involved in SFTF, what would I advise? Let me illustrate on the basis of experience. I became Director of Science Policy at the Royal Society on 8 September 2008, one week before the collapse of Lehman Brothers, and twenty months ahead of the General Election. It was crystal clear then that, whatever the outcome of the Election, we were entering a period of political and economic turbulence which would seriously threaten the funding of research in the UK. So the RS put in place a two-year plan of action.
First, we needed to gather the best available evidence, and we assembled a first-rate, broad-based panel of experts, including Nobel laureates, FFRS, former ministers, business leaders and economists (based on the model that the US National Academies had followed in their 2005 ‘Rising Above the Gathering Storm’ study). The resulting report – http://royalsociety.org/policy/publications/2010/scientific-century/ – was published eight weeks ahead of the 2010 General Election, to maximise its impact during the campaign.
Second, we joined forces with others (in public and behind the scenes) to ensure a broad-based and united coalition were making similar arguments to politicians and policymakers in the lead-up to, and most importantly after the Election. This united front included all of the academies (with the odd wobble from RAEng), the universities, the research councils, research charities such as Wellcome and CRUK, CST, NESTA and the high-profile, grassroots campaigns of CaSE and Science is Vital.
Third, we embarked on a sophisticated and carefully-planned lobbying, PR and media campaign to win the argument over the summer of 2010, leading up to the Spending Review outcome in October 2010.
If SFTF is serious about advancing your cause, I suggest you might look back and learn from this experience, and reflect on where you’re currently going wrong. Think about how you’re presenting your case, the quality and robustness of your evidence, who you are using as your spokespeople and ambassadors, and the breadth of your coalition, allies and supporters. Think about the political cycle, the points at whch you can intervene to maximal effect, and the balance between public pronouncements and private advocacy.
As I said, I’m all in favour of scientists engaging actively and vocally in politics and public policy. SFTF just isn’t doing it very well. I for one would welcome a comprehensive and independent look at all the issues that have been raised by SFTF (and others) by an expert group deemed balanced and impartial by all sides. This would ideally be framed in such a way as to rise above the specifics of what the EPSRC is it isn’t supposed to have done wrong, to look in a more comprehensive and balanced way at the balance between investigator-led, strategic and directed funding within the UK system, and related issues around prioritisation, peer review and PhD funding.
Ideally the Royal Society might play this role, but there are a number of other options. And a key issue to resolve is timing: any such intervention would need to be sensitively timed in relation to the next SR and Election, so as to resonate with, and not confuse or obscure broader messages about levels of investment in research and the links to economic growth.
I would value the response of Philip and others to this suggestion.
James
I’m afraid few working scientists I know and work with actually believe that the last CSR outcome for science funding was “steady state”, as I’ve heard some commentators and science policy specialists refer to it (I think including you?) Not in cash terms overall and certainly not when capital funding is included. As you know, CASE themselves have analysed this in great detail at http://sciencecampaign.org.uk/?p=7144 (sorry no time to learn another wiki type markup variation now!) and I’d be interested to see counter arguments to any of that. An obvious exception to this is colleagues in some areas of biomedical research, though it gets complicated by their non research council funding streams (Dept of Health as well as Charities).
This is not to diminish the carefully articulated campaign you describe ahead of the last CSR, which undoubtedly had some “impact”, but it does seem to need saying. It seemed as if most of the people celebrating the CSR “result” were working in science policy or journalism rather than science research constituencies…not least as the former are so much better at organising themselves(!) Would this hard work have succeeded without some of the higher profile and more immediate “Science is vital” type campaigning to go with it? You ideally want a combination of the two which I guess you’re saying is what you worked together to achieve…
Others may feel when you look at the bigger international picture, which scientists inevitably do through their work, it could have been much different. How or perhaps when it might have been paid for is another debate. Hence if the tone strays into lecturing about how successful (presumably in the very short term at this juncture?!) the SFTF approach is then they may understandably call that into question.
Athene
I’m not sure it’s advisable to try to direct a comments feed such as this in the same way as I expect most scientists don’t believe it is advisable to try to steer fundamental research! As others have pointed out, you have kind of made the case for the often unexpected nature of impact itself?
I wonder if SFTF studied what happened with STFC funding (ahead of the economic crash) and the mixed modes of protest, at times appearing to be suppressed within the community, and perhaps came to the conclusion that quiet persuasion alone was not sufficient? That’ll get Mike M back here again?! 😉
Jonathan,
Enjoyed your comments!
The suggestion that a comments thread in a very public debate should be ‘steered’ in a particular direction indeed has rather interesting parallels with the idea that there are those who know best – or better than the rest -about the directions that scientific research should follow…
And, yes, there are certainly a number of researchers associated with SFTF who are very aware indeed of the STFC debacle and just how protest was ‘managed’. Peter Coles makes the point at his excellent Telescoper blog .
James,
Sorry for taking so very long to respond and thanks for your comments.
We disagree fundamentally on the scale of the problems with EPSRC. Shaping Capability; no resubmissions; the frankly unethical blacklisting scheme; the award of fellowships only in specific areas defined by EPSRC; the move towards a much more opaque funding mechanism (a la the recent “New Directions” tranche of funding for fellows and the “Dream Fellowships” scheme, for example); EPSRC’s move from funded to sponsor; and, of course, the requirement that all research projects – be they in near-market R&D or fundamental quantum physics – be driven by a consideration of impact: these are not minor issues. They represent a dramatic change in how science is funded in the UK.
Moreover, make no mistake – what happens at EPSRC will influence and inform developments at the other research councils. These are not ‘narrow issues’ confined to one research council. What happens here with respect to EPSRC will set in place the directions all research councils make in the future. (As you’ll know, for example, David Delpy is the RCUK Impact Champion).
Jonathan Tedds’ comments re. Science is Vital are very well-made.
Where I agree entirely with you is on the matter of establishing an independent, preferably international, panel to review the policies and processes of EPSRC. As Chris Hayes points out below, this precise recommendation was made by the panel for the last International Review of Chemistry (coordinated by EPSRC). EPSRC’s response was to say that no action was required.
PS My apologies for exceeding the 4000 character limit!
Although the word is clearly loaded, I don’t see any problem with using a stunt to make a point with politicians. When Richard Feynman dunked an O-ring in ice water at the Challenger Enquiry, it was clearly a stunt rather than a controlled scientific experiment, but it made the point succinctly and effectively.
As you say, politicians are not scientists for the most part, and generally do not have the time nor the interest to engage in a thorough scientific debate or even a “reasoned presentation.” They just need a gut reason to support science, and a good stunt like the Science is Vital protest of silly “up and atom” placards and massed ranks of lab coats gives them exactly that reason.
So, my problem with this event wasn’t that it was a stunt. It was that it was such a bad stunt. I was deeply embarrassed that this was the best they could come up with, as, I suspect, were the politicians it was aimed at. If you insult their intelligence by setting up such a nonsense in the belief that it will persuade them of your case, then it isn’t such a surprise that you end up losing what support you had.
..and yet, Mike, as I say to Owen above (who rather less politely makes the same point as you), the feedback from MPs is that they could very much appreciate our concerns regarding EPSRC policy. They were particularly troubled by the lack of transparency.
Regarding gauging support: You seem to be very confident that the stunt last week led to the ‘dissipation’ of a significant amount of support for SFTF. The “Twittersphere” and the “Blogosphere”, however, aren’t the best indicators of the strength of support in this case. Those who make public statements of support for EPSRC via Twitter have nothing to lose. Those opposed to the new EPSRC policies certainly perceive that they will have something to lose. (I can forward you the e-mails I’ve received which make this point very clearly, if you’re interested – with all attributions removed, of course!).
This is not a criticism of EPSRC. As Richard has pointed out, I have not ‘suffered’ in terms of funding losses from my vocal criticism of EPSRC’s policies. But there is clearly a very strong perception amongst very many EPSRC-funded scientists that if they raise their heads above the parapet their funding prospects will, at best, be compromised.
See you in the tea room at Nottingham to discuss this further? 🙂
[Godammit. Well over the 4000 character limit…]
I merely voice my own reaction to the stunt, and extrapolate to how others might react to it: I was embarrassed by the cheesy nature of it, and suspect that many politicians would share the sentiment.
The feedback received from MPs may well share the concerns, but I am sure you are as wary of selection biases as I am: a less excruciating stunt could well have brought on side many of the less engaged politicians, and contributed to the political groundswell that is clearly needed, rather than polarizing opinions and turning off the not-yet-interested.
Let me stretch Athene’s patience by returning to the subject of impact. If the meaning of the “impact agenda” was that all research needed to produce direct short term economic impacts, or to compel every scientist to produce protectable intellectual property then I’d be with you opposing it. On the other hand, I completely subscribe to Steve’s eloquent comment above – I do think we are in difficult times (thinking globally here), and I strongly believe that new science will be important, indispensable indeed, in dealing with those difficulties, so if the “impact agenda” is concerned with helping science cash its promises, then I’m right behind it. The truth is that “impact” isn’t a thing, it’s a dialogue, a negotiation – at the risk of annoying Athene by using her blog to advertise my back catalogue, I wrote about this aspect of impact at length here.
I’m quite sure that there are people in the Treasury who think the only impact that matters is that we get another 0.2% on next quarter’s growth figures. It’s to the credit of RCUK that by the time they wrote their definitions of impact that’s been broadened to include many non-economic factors such as “enriching public engagement and the quality of life”, “environmental sustainability”, “public engagement with research”, as well as indirect economic effects such as “enhancing the research capacity, knowledge and skills of public, private and third sector organisations”. It seems to me that opponents of the “impact agenda”, by interpreting it in the narrowest economic terms, immediately concede the most important ground, rather than taking the opportunity to say, yes, science does have an enormous impact, but it manifests itself in more complicated ways than some people think.
It’s interesting that BB raises the case of synthetic biology. SynBio is a field whose rhetorical construction has been based almost entirely on the idea that by taking an engineering approach to biology practical results will quickly follow, inevitably described in terms such as “a new industrial revolution”. Here the promise of impact has come from the community promoting this field, not from outside at all. I think SynBio is also connected with the phenomenon BB identifies of high profile papers increasingly valuing “we made a nano/bio-widget” papers rather than papers about understanding. I agree with BB that developing that understanding can be just as important in the long run for creating valuable applications than making another nanowidget … but this again is talking about the complexities of the way science contributes to innovation and growth, rather than doubting the premise that it should so contribute.
I’d certainly agree with Philip we need an evidence base for science funding policy, and that there is a lot of ideology around. The tradition that Philip draws heavily on – Merton, Polanyi, Ziman – was rooted in a particular time and place – the West in the Cold War. I think it was naive about the realities of science at the time, and isn’t easily transferrable to the state we’re in now. I know the Salter/Martin work too – Ben Martin was on the Scientific Century working group. The conclusion I think I draw is that it makes no sense to say science can make this or that contribution to economic growth in isolation from considering the broader innovation system it sits in. That we are having this discussion is in a sense a symptom of the fact that that our innovation system has been transformed in the last 20 years, in ways that are not widely recognised. We have to do the best we can in the circumstances we’re now in.
I certainly agree that it would be appropriate to base funding decisions on the long term societal impact of research IF the long term societal impact of the research could be accurately predicted.
In the case where the long term societal impact of the research CANNOT be predicted (and I would argue that the past history and recent history of science would indicate that this is indeed the case we find ourselves in) then I think it is entirely inappropriate to base funding decisions on an essentially baseless criteria.
So once again, is there any evidence to suggest that either peer review or the EPSRC are actually capable of predicting the long term societal impact of a given piece of research?
I promise I will change my mind utterly about the value of impact statements if you can point me at some such evidence!
oh yes and on the topic of synbio, I sat through 3 talks in a row at a conference a couple of years ago about using a particular virus as a delivery tool. Each group had started from the same point and just done what amounted to trial and error until they got what they were looking for.This was being spun as a ‘fast and efficient’ way to develop new tuned delivery systems.
I asked a question at the end along the lines of “wouldn’t it be a lot more efficient if you actually understood how the viral capsid proteins worked and could leave out the trial and error phase?” It was met with blank faces and a restatement that they were already using the faster more efficient trial and error method than their predecessors trial and error method.
Is it easier to get funding to make a new drug delivery tool or to understand how viral capsid self-assembly works?
Richard,
I really must attend to the ‘day job’ but, as ever, feel the need to respond to Richard’s excellent comment.
Yes, there is much about Ziman, Polanyi, Merton et al. which is rooted in a particular time, geography, and ‘socioeconomic climate’. I agree entirely.
But are you really saying that the disinterestedness norm proposed by Merton is something that we should discard in the 21st Century? Do Ziman’s arguments regarding the extent to which the scientific method can be compromised by ‘instrumental considerations’ (to use his terminology) not make an impact on you at all?
.And it’s not just Merton, Ziman et al. I’ve read. I’ve referred repeatedly to Mirowski (to whom James Wilsdon also refers above). It’s also worth referring to Helga Nowotny’s work (Mode 2 etc.., of which you’re well aware, I know). Compare and contrast Nowotny’s support for the importance of basic science (via the European Research Council) with EPSRC’s disjointed and inconsistent messages about impact and how it can be measured.
Putting aside the ideological objections, where is the hard evidence that directed research in the physical sciences (before Atti jumps in to give me some statistics on engineering!) produces greater levels of innovation than exploratory research? The ‘elephant in the room’ is, of course, which robust metrics exist to be able to compare outputs from the two modes of research? Time-scales are different; closeness-to-market is different; potential for disruptive technologies are different…
As Salter and Martin point out, it’s effectively a fool’s game to attempt to directly quantify the economic impact of basic science.
[And did I really say ‘elephant in the room’? Christ. It really is time for more caffeine (followed by a sound thrashing)]
Wow. Posted my comment seconds after BB and find that we make rather similar points. Show us the evidence! (…which, for the reasons Salter and Martin and many others (e.g. Paul David) point out, doesn’t exist in any type of credible form. Bit of a catch-22 there.)
Oh dear – Athene is going to be cross with us…another 3 pages of impact!
But “impact” isn’t about predicting the future! The guidance from RCUK is absolutely explicit about that:
“At the application stage we do not expect applicants or peer reviewers to be able to predict the economic or societal impacts that research will achieve. However, we want to encourage applicants to consider and explore, in ways that are appropriate given the nature of the research they are proposing to conduct, potential pathways to impact, for example through engagement or collaboration with partners.”
You may doubt that research councils, reviewers and panels will hold to that principle but it’s only by engaging with “impact” that you can hold research councils to it.
Philip, the Mertonian norms may be admirable, but they certainly didn’t apply in 1942, if they applied in the 50s and 60s it was only to a tiny minority of scientists in academia (who were supported by the state as a by-product of the very non-Mertonian contributions of their colleagues in the military-industrial complex), and they don’t apply in the circumstances (of political economy) we’re in now. If you want to smash today’s neo-liberal hegemony, I’m probably with you, but alas I don’t think that’s the job of the research councils.
The discussion of whether we should have directed research or exploratory research is a false dichotomy – of course we should have both; the issue is the balance and relationship between the two. The ERC is a marvellous organisation, but of course it sits in the context of a much larger directed research effort in the Framework program. In the 1950s and 60s in the UK and USA, a few academic scientists worked in considerable freedom (and indeed some in industrial labs like Bell had quite a lot of freedom too), but this was the tip of a vast iceberg of applied research in government and corporate labs (the elephant in the room at the time was probably balanced on the iceberg’s tip). One of the reasons this is such a pointed issue now is that this balance has hugely shifted, to the extent that in the UK now about 30% of all research is carried out in universities, a proportion that has pretty much doubled in the last 15 years (largely as a by-product of the aforementioned neo-liberal hegemony).
Thanks for that clarification (also I read some of your other blog entries). It is starting to make sense….
I don’t understand how whether or not a scientist has had a good think about how to funnel their research outputs towards future applications can be a criteria to decide on fundability? Maybe what you are trying to say is that it isn’t used that way.
Would it be fair to say that it is the engagement with the process of thinking about how your research may led to impact that is important rather than the actual likelihood of the things your musing produces actually making it into reality that is important?
Is it a yes/no thing? This person has thought/not-thought about it? Rather than a this persons ideas for a spin out company are slightly less far fetched than the other persons as a continuum assessment?
If what you are saying is really the practice, then there are a whole lot of senior people around here who either don’t get it or are deliberately giving bad advice to us “youngsters”.
Either way, thank you so much for giving me a valuable new perspective on this…..it will at least make the next pathway to impact I write feel slightly less like sawing my own arm off…
Richard,
At the risk of attracting Athene’s ire for the cardinal sin of quoting from an academic paper,
Perhaps I’ll head off Athene’s ire at the pass if I include some numbers from the same paper?: Subscription to the Mertonian norms ranged from 73% to 91% amongst US scientists among US scientists.
[Source: Anderson, MS, Ronning, EA, DeVries, R and Martinson, BC. J. Higher. Educ. 81 366 (2010)]
So you may well argue that the Mertonian norms (Ziman’s ‘Legend’) do not apply. But from the evidence in Anderson et al.’s paper, very many scientists certainly consider the principles espoused by Merton to represent the essence of the scientific endeavour.
Are those principles/norms challenged by other sociological factors and by the prevailing socioeconomic environoment? Of course. And yet, as Anderson et al. point out, scientists still widely subscribe to Merton’s norms. In the 21st century.
You say:
“The discussion of whether we should have directed research or exploratory research is a false dichotomy – of course we should have both; the issue is the balance and relationship between the two. ”
Who could disagree with this? (Despite Athene’s perception of my stance on impact.) Your example of the ERC in the context of the overall EU Framework Programmes is very well-chosen. Note how the ERC budget is increasing. Note also how the ERC embeds impact in its funding programmes. It sets up a spearate fund for those interested in impact and asks those interested to bid into that fund . It does not, as EPSRC does, aim to change the very bedrock of basic science by coercing academics into making a consideration of (socio)economic impact a core aspect of their research programme.
By imposing the contraint that socioeconomic impact must be incorporated for all areas of science it funds – be that esoteric mathematics/mathematical physics (see, e.g. Martin Rees’ questioning of John Armitt during the Sci & Tech. Select Committee session late last year) or fundamental nanoscience – EPSRC entirely skews the balance away from exploratory research. And so the balance to which you point is fundamentally undermined.
And I can’t agree that a consideration of impact does not require prediction. The statement from EPSRC you quote at the head of your comment simply doesn’t ‘scan’. It’s a non sequitur whatever way you look at it.
I’ll put the same question to you Richard, as Lord Rees put to Armitt. Say that I’m a mathematical physicist working on an esoteric problem in quantum field theory. Please tell me how I write not only the Pathways to Impact statement but the National Importance component of my grant application.
I’ve watched a couple of EPSRC’s programme managers flounder on this. Armitt failed spectacularly to address the question. I’m confident that you’ll do a much better job.
Best wishes,
Philip
Responding late to Philip Moriarity’s post at 2 p.m. today, you may be interested to hear that EPSRC have invested publicly-funded staff time in monitoring the twitter feeds of people saying critical things about EPSRC policy, and I, for one, have had a visit from my Head of School to rap me over the knuckles because he had a cross phone call from ‘someone senior at EPSRC’ about something I said (and, moreover, only in reply to someone else) on Twitter. So you’ll perhaps pardon some of us if we seem a little paranoid.
That is absolutely and utterly beyond belief, Simon.
I am going to send Atti Emecz an e-mail about this right now. I’m sure he’d like to put across EPSRC’s side of the story here…
Philip
Simon and Philip – Unfortunately this is not an isolated incident. I know of other cases of EPSRC contacting Heads of Departments (and one Vice Chancellor) because of comments made on Twitter.
Some of my colleagues were worried about attending the SFTF lobby event on May 15th as they had been told that EPSRC would be monitoring attendance. I don’t want to name the individuals involved, but I can forward them to you in confidence if you need further examples for your conversation with Atti. There are a lot of worried people out there who are genuinely scared to speak out in public. This cannot be right.
Well, to be fair, what I had said was pretty intemperate and more said in a spirit of frustration than after calm reflection, but even so, I was surprised to get the visit from my HoS, incidentally some six weeks after the offending tweets. Frankly I felt that EPSRC staff time could be better spent. I know I am not the only person that this has happened to. In one case I believe the rap over the knuckles was administered by the person’s VC…
Good point Philip. The “Pathways to Impact” clause is fundamentally incompatible to the higher requirement of scientific method. This is not for scientists but office works and civil servants in research departments. And to be written down upon preliminary results only. A research policy driven by “Pathways to Impact” would run the risk of reproducing a lot of Trofim Lysenko scientists.
The original post seems to have made quite an impact!
Unfortunately (at least for keeping up with my own blog) I have spent most of today on science policy, as it happens, and haven’t been able to do more than moderate comments from newcomers as they join in this discussion. Interesting how much my ‘ire’ is invoked.! I promise you it’s not on the loose. I had felt the comments were rehashing familiar ground and hence I tried to steer the debate away from impact, but I am delighted to see later comments go in so many interesting directions, historical, European and ‘freedom of twitter expression’. I hope Atti can reassure us when he has had a chance to talk things through, but Simon’s further comment indicates that the tweet in question was not connected with the recent SFTF event, but some other event that exasperated him.
It’s good if BB has been helped to understand the economic interpretation of impact alone is too narrow. I apologise to Richard for not citing his excellent earlier blog on the different meanings of impact in my original piece. I had wanted to, but felt that my post was already long enough without bringing in those ideas. Clearly a mistake. One fundamental confusion which I think underlines the debate above is a possible mix-up in some people’s minds between Pathways to Impact statements – which are of the broader, reflective kind – and the REF Impact Case studies which deal with what has happened already and how to measure that impact in terms of significance and reach. I can assure you that measuring that is pretty nigh impossible in most cases, even though it’s looking back (I was on the Physics Pilot panel and I know how we struggled). I don’t think anyone has ever suggested predicting quantitative measures for the future in the Pathways to Impact statement.
I do wonder how many commenters have sat on panels which have dealt with grants with Pathways to Impact statements included. Some have I’m sure, but I do feel if you look at what happens you might feel reassured that impact is meant to cover all the broad types that Richard listed, and it is a way of demonstrating you aren’t simply taking Government money and running but you have thought about why you’re doing it. And that in itself does not preclude pure/basic/blue skies – whichever adjective you care to use – research. I am sure Richard would do a better job of answering Philip’s challenge, but I would have thought someone researching mathematical quantum field theory would appreciate that many schoolkids are fascinated by quantum ideas, so they’ll go into school to fuel their curiosity about the quantum world. Perhaps they can think about how to construct some simple cartoons on Schrodinger’s cat or ways of explaining quantum tunnelling. Maybe they can join a local science festival to talk about it. That strikes me as utterly obvious stuff to attempt. Perhaps they can do more and discuss how likely quantum computers are to be usefully viable anytime soon and what it is that are the current stumbling blocks (maybe even digress into ethical issues about privacy, cryptography etc). That will in no way impinge on their own research direction, but it will use their experience and knowledge to share the excitement of what they do with those less mathematically expert. Sure it will take a bit of time, but one would hope they’d want to disseminate their love of the subject to future generations. How does that ‘erode the scientific method’?
We do need to keep a sense of proportion about impact. EPSRC have stated research excellence remains the main criterion. Impact is a small factor in decision making in my experience (although, I will say again, most of that does come from BBSRC committees but that shouldn’t make a significant difference), and even then it is about far more than widget making and certgainly does not equat to guesstimating some unknowable future ‘value’ of the research.
I am sure there were many other valuable points I might address if it wasn’t late and I have paperwork to read for tomorrow’s meetings (which, I should add, includes having dinner with an MP to talk about education matters; I am putting my mouth where my money is and engaging with politicians). I have wildly exceeded 4000 characters no doubt, but since everyone else is breaking any attempt at ‘rules’ on my part I may as well join in! I do appreciate all this debate, whatever my initial attempt to prevent a simple revisiting of familiar arguments may have looked like.
Athene,
I’d value reading your response to Mike Duff’s very important comments and questions re. EPSRC’s perception of just what impact means. I hope that you will be able to find time to respond. I certainly appreciate just how much this debate can eat into time for the ‘day job’ but we’re publicly funded, after all, and so I feel that we’re obliged to engage publicly with the arguments.
Your response to my question about the Pathways to Impact statement for fundamental quantum field theory is, I’m afraid, entirely as expected and is not at all convincing.
I seem to recall that you were highly critical of Pathways to Impact statements which contained nebulous ‘generalised’ statements. *Anyone* could write the Pathways to Impact statement you’ve suggested. As you say, it’s an entirely obvious approach. That’s entirely the problem.
Moreover, a researcher may well be a truly awful public speaker, have the telegenic/photogenic appeal of a sheet of wet newspaper, and be the last person we would want to put in front of the general public to promote science. But they also could be capable of ground-breaking science. I can certainly think of, for example, world-leading scientists who I wouldn’t ever want to put in front of primary school children for fear of putting them off science for life. I’m sure you can too.
The rather feeble EPSRC response to this is that well, that person doesn’t have to do the public engagement, they can get someone else to do it.
But why do you (and EPSRC) think that the best way to inspire future generations is to coerce all researchers into writing public engagement statements simply to tick the ‘impact’ box, rather than to support – with, for example, something like the now-defunct Partnerships for Public Engagement scheme – those who are truly enthusiastic about outreach? (Great scheme that. I’m thoroughly enjoying my involvement with a PPE project – Giants of the Infinitesimal – along with one of Richard’s colleagues at Sheffield, Ash Cadby. Interesting to note that if I still submitted EPSRC proposals, I’d have been blacklisted for two public engagement proposals which ‘went down’- a rather novel EPSRC strategy to encourage speculative approaches to new types of public engagement.)
As for the quantum computing ‘spin’ you suggest? If I still refereed EPSRC proposals I would not score that particular idea very highly as it seems like you’ve desperately tried to shoehorn in a link to technological development which is entirely unconnected to the objectives of the grant. “Opportunistic” and “unconvincing” are the terms that spring to mind.
You say: “… one would hope they’d want to disseminate their love of the subject to future generations. How does that ‘erode the scientific method’?”
I really don’t get your argument here – you’re conflating two entirely distinct points. My concerns about the erosion of the scientific method are laid out pretty clearly in the comments stream above (and elsewhere) and relate directly to RCUK’s #1 hint on how to write a successful Je-S application. Please tell me how the impact programme you lay out is meant to *inform* the direction of the research, as RCUK recommends it should for a successful grant application.
Finally, as Mike points out, what draws very many students into physics are the “big” fundamental questions of life, the universe, and everything (if you’ll excuse the tip of the hat to Mr. Adams). The evidence for this is available from the IoP, as you no doubt know. It therefore seems just a little ironic that you are arguing for a more instrumental/utilitarian appraoch to science (via the imposition of the “Pathways to Impact” statement) as a means to inspire the next generation.
Admittedly having only scanned some of the later comments, I am still confused as to the arguments for not explaining your potential impact (whether cultural, economic, health) and how you will maximise the chances of that happening when you write a grant application. Can anyone explain this (concisely) to me?
In times of austerity, and with major scientific and economic challenges facing society, should we not have to justify why our pet projects are worthy of receiving significant amounts of public funds, particularly in competition against other grants also seeking funding? I often find in such conversations that there is an underlying arrogance to the impact is irrelevant argument (and this is certainly not a comment aimed at Philip). The argument of many is often to take a stance of the superiority of pure, blue skies science vs the tainted research of projects with envisioned possible applications (which Philip refers to as R&D). I think this is partly what drives a divide between scientists on this subject. I don’t see why, if your research has an envisioned prospective use or application for society, you shouldn’t receive more backing from public funds than those who have been unable (or unwilling) to consider what the future value of their projects might be.
[lights blue touch paper and stands back!]
It would make perfect sense to only fund the science with the greatest benefits to society. Do you think you have a way to predict what that science is? I am sure everyone (including the EPSRC) would appreciate knowing your method!
There are so many examples in the past of a completely unexpected scientific discovery having a huge impact in an unrelated field.
Take the energy crisis, we could focus all our funding on what seems to us the most likely routes to success and completely miss the solution which would have come from an unrelated seemingly application free area of blue skies research.
When science is close to producing applications already then it makes sense to discuss which applications are more valuable or closer to success.
But for long timescale blue skies research I am of the opinion that it has not been demonstrated that directed research is more effective than simply funding good people to do interesting science for the hell of it.
BB, you misrepresent the point I try to make. Why should EPSRC play a lottery of randomly guessing what might make an impact. Surely those researchers who have considered what the impact might be, and how they will aim to achieve it, will have a greater probability of achieving impact.
And of course I refer to impact in its broadest sense of not just economic and health, but cultural enrichment etc. Could we not expect pure blue skies scientists to demonstrate more of their work at science festivals, in schools, in public lectures? Is that not impact and cultural enrichment in itself?
I think it is actually worse than you say, as it is not just a waste of time and effort to pursue this agenda due to the impossibility of second-guessing impact. It actually creates an unhealthy culture for both scientists and industry, which, perversely, will reduce the impact of science rather than increasing it in both the short and long term.
No matter what wording you put in seeking to dilute impact to a point where the cultural benefit of blue-skies research counts, you will always end up skewing the research agenda: in the climate being created, a scientist with two ideas, one with a very high risk but potentially revolutionary in a rather vague way, and the other with much lower potential but a clear pathway to impact, is now much more likely to pursue the latter to tick the boxes with the bosses. But i would argue that it is the former that should have by far the greater call on public funds, as it is that kind of research that leads to the step changes that change the World, which there is no other way to fund given the risk.
And as well as this long-term strategic damage, there is also associated short-term tactical damage: if companies take even more the view that it is not their job to develop new near-market ideas because the Government has taken on funding that for them, then you create a “research dependency culture” that suppresses innovation and research investment in the commercial sector, and we end up with far less near-term technological benefit to society over all.
What Mike said.
(Character count:15).
Philip
Surely “potentially revolutionary” is indeed impact, if framed correctly. Many arguments are based on the fact that lasers and x-rays were discovered without thinking about possible applications. But in those days they weren’t required to think about impact. And if they were, would applications have been found sooner? And how much “blue skies” data has “dropped down the back of the sofa” and missed the opportunity for translation into revolutionary new technologies because the researchers were not made to consider this? If a researcher does not have the vision to see and think beyond the project they are proposing, then is that person worth funding?
All contentious points, but I don’t see how being made to consider how your work might fit into a bigger picture in the future (and making a plan to exploit that) is damaging to pure research.
I am happy to honestly write that all of my research is “potentially revolutionary,” because “potentially revolutionary” actually means “most likely will have no use whatsoever.”
Are you saying that research into lasers should not have been funded unless the researchers had the clairvoyance to predict that one day we would all carry bits of plastic instead of money and that people would try and forge those pieces of plastic and that one of the most effective anti-forgery measures would be to use a laser to create a three-dimensional image of the company logo on the card? Or that they should have written an amazing tale in their impact statement that one day everyone would have a computer on their desk, and that software for that computer would be distributed on shiny disks about the size of a coaster, which would be read at implausible speeds using a tiny laser that costs a few pence?
If they really could make such predictions with any degree of reliability, they would be very rich indeed. And their research should be funded by MasterCard or Sony.
Michael, it does sound ridiculous when you take the concept to such absurb extremes. However, there’s a huge difference between vision and considering a pathway to impact, and clairvoyance.
Hi Paul — you asked specifically about the laser, which is why I wrote about the complete unpredictability of its real impact at the time the research was proposed. I could happily do the same about X-rays, or particle physics in the context of the WWW revolution, or many of the other revolutionary discoveries we would miss out on if their cases had required to be strongly informed by an impact statement. Indeed, the fact they are revolutionary almost by definition means that their impact could not be predicted.
My argument was absurd, but that is the nature of reductio ad absurdum: when confronted with a way forward that leads to a logical inconsistency when pursued, you know that it is not a good pathway to follow.
I have no problem at all in asking all grant applicants to consider the possible commercial impact of their research. I just take the view that a resulting statement along the lines of “I have looked into this and can see no current commercial uses for this research” is actually a reason why the research councils should be supporting it, rather than a reason why they shouldn’t.
Athene, as one involved in mathematical quantum field theory I endorse everything you say about public engagement. When I talk about it in schools, or the Royal Institution or occasionally on tv and radio, I never cease to be encouraged by the public fascination with the wonders of the quantum world. It is one of the main reasons young people are attracted to science in the first place. Alas, this not what EPSRC’s Impact Agenda is all about, at least not according to Director of Research Lesley Thompson. At a talk in Lancaster, she made it clear what they were looking for:
“If you are in involved in basic mathematics: will it revolutionise engineering?”
“Will you have an impact on the quality of life in the UK?”
“ Will you have an impact on health care?”
“Will you spawn a whole new sector of the economy?”
I had to suppress a chuckle when she concluded
“One thing for sure; we are not asking you to tell the future”
So which view of blue-skies research prevails in Swindon? The enlightened scientific one of Athene Donald or the commercially driven one of Lesley Thompson? Well I am sorry to tell you that EPSRC, without any consultation with the community, last year wiped Quantum Field Theory off the Mathematical Sciences Portfolio. Grant proposals in this area were consequently rejected by the non-scientists in Swindon without peer review. When I asked your friend and mine Atti Emecz why the community was not informed of this change in policy, he responded “The remit of our mathematics programme has NOT changed. The Maths programme does not fund theoretical physics”. I would disappointed if a fellow physicist such as yourself finds this acceptable. If you don’t, what non-political mechanisms do you have in mind to change it? Please don’t say “dialogue”, it hasn’t worked.
In order to offer a different perspective on the SFTF campaign, I’d like share my views with you all on why I attended the lobby event on the 15th of May. For the record I am an organic chemist, and I have read the posts appearing on this blog with great interest (Athene – many thanks for hosting this, you now seem to have created THE hub for discussing this issue openly. Fantastic).
I have kept quiet so far as my field of research has been identified as one to be ‘reduced’ in the EPSRC Shaping Capability exercise, and I’m aware that any comments left here, can (and probably will) be dismissed as complaining about cuts. Here we go anyway.
First, a good news story about ‘Impact’. My School of Chemistry has just received a £12 million pounds donation from GlaxoSmithKline (GSK from now on) for the construction of a new building that will be a centre for Sustainable Chemistry. This is fantastic news, and we have had some great meetings with GSK to discuss what challenges face the Pharma industry, and to identify what new chemistry needs to be developed for the industry to be sustainable (in all senses of the word) in the future. This is the largest single donation to my University, and it has certainly made our VC take notice of the good work that we are doing in Chemistry.
Why did GSK make this donation? Well, here is what Sir Andrew Witty said in his BBC press release of 30th Nov 2010:
The company’s chief executive, Andrew Witty, said: “We’ve invested in Nottingham because it’s one of the best universities in Britain, in fact the world, from a chemistry point of view.
“In fact the majority of our chemists who we hire into GSK come from Nottingham so we have a long history there.
“There’s a great record of excellence in that school and they absolutely deserve that investment.”
It is clear form this that GSK value the people that we train, and they recognise that we are doing a good job. Fantastic right? What more could we be doing to engage with the health and well being of the UK?
The EPSRC 2009 international review of chemistry was very supportive of UK chemistry, and it was particularly complimentary to Synthetic Organic Chemistry. I don’t want people to glaze over, but ‘we’ (i.e. SFTF supporters) have been challenged to provide evidence for our case, so here are a few quotes from the report (which is published on the EPSRC web site by the way if you want to read the full 72 page document):
Historically, organic chemistry has been a bright star in the constellation of UK chemistry, and the area, particularly synthesis, remains in a position of notable strength. It is a discipline that occupies a core position and at the same time has unlimited potential in bridging to a variety of other fields. Thus, organic chemistry (synthesis) can find resonance in inter- and multi-disciplinary research programmes, such as those in analytical, materials, biological and medical sciences.
The report then goes on to say:
Numerous cases could be identified in which fundamental discoveries and observations in organic chemistry have been translated into successful spin-off (start-up) companies, resulting in wealth and job creation in the UK
and….
Additionally, there are programmes in catalysis, methodology and natural
products chemistry that are globally competitive. The Panel heard consistently repeated statements from the fine chemicals and pharmaceutical industry of the high value they placed on the trained PhD organic chemists that they recruited in the UK and of the interdependence of each party on the strong collaborative arrangements in place.
All of this seems to be what government want scientists to do. Right? So can anyone explain to me why EPSRC funding for Organic Synthesis is going to be reduced? Why have PhD students been removed from responsive mode grants, why is there only One DTC (10 students p.a.) in synthesis in the UK? When pushed on this matter, Prof Delpy cannot adequately justify it. What the EPSRC web site says is that there was a ‘spike of funding’ in 2008/2009 and funding at this level is not sustainable. Well lets look at why there was a spike of funding in 2008/2009 for Organic Synthesis. It is largely due to two targeted initiatives: (1) Array Chemistry and (2) Flow Chemistry. BOTH of these were run in collaboration with industry (mainly GSK and Pfizer), and both were seized upon by academics as new sources of funding. What I’m trying to say is that scientists engaged with these areas, and they tried to work in areas that had ‘Impact’, as directed by INDUSTRY needs. It seems very bizarre to all of us in this area that our funding is going to be reduced BECAUSE we tried to show impact. Can anyone explain that to me? Prof Delpy, are you listening?
Behind closed doors, I think that EPSRC are using Journal impact factors and citation rates as the primary measure of Excellence. They pretty much say so in a few published documents, but they are not very explicit. This takes me back to an earlier comment made by Athene regarding the roles of journals in all of this. Basically, I can understand that Prof Delpy wants all EPSRC-funded research to be published in Nature, Science etc. Why wouldn’t you. The fact is that these journals are not entirely representative of the vast array of activity that EPSRC fund. How many Engineers regularly publish there? In my area of science, Nature Chemistry is the highest impact factor journal, and we are encouraged to send our work there for publication. We do this for REF purposes, but we now also need to do it so that Prof Delpy doesn’t cut our funding. This seems a bad way to go, as Nature Chemistry publishes about 100-150 papers a year (someone will correct me if I’m wrong), and this gives only a very thin slice of the total chemistry output. Some great papers are indeed published there, but some quite average ones (my subjective opinion) are as well. Some (many) great papers are published OUTSIDE these journals, and we must not dismiss the fact that high impact work (judged in retrospect) can be published in a lower-impact journal.
I am very worried about all this, and although I look at the ‘Impact’ issue from the other side of the fence from Philip Moriarty, I can see that it may be possible to have too much ‘industrial impact’ and not enough ‘journal impact’. I think that Philip is objecting to the notion that you MUST deliver impact in your research. He is NOT saying that you must AVOID impact at all costs. He is simply saying that we should all do the best science we can and let users of the research deliver the REAL impact. If we stray from this we are all in danger of operating in science fiction rather than science fact.
Back to SFTF and our issues with EPSRC (not Government, by the way). The symptoms are all different. The issues with EPSRC vary depending on whether you talk to a chemist, a physicist or a mathematician, but the underlying cause is the same, and that is the very heavy-handed top-down attitude of the senior EPSRC management. There is a ‘one-size-fits all’ approach to EPSRC delivery, and there is no allowance at all for the fact that Engineers might need something different from Scientists, and that Chemists might need something different from Mathematicians. There is not enough flexibility in the system, and an APPLICANT should be allowed to apply for whatever resource they want, with their obligation being to justify the resource requested in order to satisfy a peer-review group (not EPSRC administrators). We should be allowed to apply for funds to deliver the best quality science, whilst also being mindful of the current need for ‘efficiency’. I just want the biggest scientific bang-per-buck for UK tax payers. Current EPSRC policies will not ensure this.
I’ll stop there, as I’m well over my word limit, but I would welcome a real open, honest (and robust) discussion on this. I only hope that the EPSRC will properly engage.
Towards the end of an earlier post (May 22nd, 12.51), James Wilsdon said:
I for one would welcome a comprehensive and independent look at all the issues that have been raised by SFTF (and others) by an expert group deemed balanced and impartial by all sides. This would ideally be framed in such a way as to rise above the specifics of what the EPSRC is it isn’t supposed to have done wrong, to look in a more comprehensive and balanced way at the balance between investigator-led, strategic and directed funding within the UK system, and related issues around prioritisation, peer review and PhD funding.
and I totally agree, and this is one of the things that I hope my support of SFTF will achieve, and for the record, this review was also called for in the 2009 EPSRC international review of Chemistry action plan (page 16 of the EPSRC action plan document if anyone is still out there). EPSRC responded by saying “No Action Required“.
Discuss…….
We should be allowed to apply for funds to deliver the best quality science, whilst also being mindful of the current need for ‘efficiency’. I just want the biggest scientific bang-per-buck for UK tax payers. Current EPSRC policies will not ensure this.
I think that’s the nub of the matter.
Given that there is a limited budget and very good grants cannot get funded, hard decisions have to be made. Whether those decisions should be made on the basis of ‘impact’ (however defined), by an RC defining the areas they are and are not going to fund, or simply by flipping a coin, I remain to be convinced that preemptive shark-jumping is the way forward.
I hope Athene does not mind my using her blog for this post. As he indicated in a post, Philip contacted me last night to ask for a response to some of the earlier posts regarding EPSRC contacting univeristies. I did provide Philip with an inital response but thought a more public response should also be made. I hope the bullet point format below is not too “formal” in style.
– EPSRC respects the right of individuals and organisations to criticise, challenge and/or object to our policies. We have said so publicly on numerous occasions. What we do not tolerate is abusive or other inappropriate behaviour directed toward our staff. We have a duty of care as an employer and we take that seriously. A statement explaining our position is on our web site http://www.epsrc.ac.uk/about/standards/employment/Pages/supportingourstaff.aspx
– Having talked to all the EPSRC executive leadership team (bar one individual who is on annual leave) I can confirm that EPSRC’s interactions have been fully in line with the above policy.
– EPSRC does not have a formal function or process to monitor twitter feeds although some of our staff are twitter users. We do have our own twitter account run by the central communications team and this is one of the many communications channels we use. There is no formal “monitoring” of twitter or indeed any other social media (including blogs) but staff may become alert to EPSRC-related communications either as part of their work or through their personal use. Where this happens we may respond. This post is a case in point.
Firstly let me say I have found it interesting to discover how widely this dialogue has been followed – not just amongst the usual blog-readers or even the EPSRC but further afield. Readers may also be interested to know that Adam Smith is planning a follow-up article on Impact in the Guardian sometime soon (Richard, Philip and myself are, to my knowledge, being interviewed, I don’t know who beyond).
Philip has laid down some challenges to me (and indeed others). Would that I were only trying to do the ‘day-job’; this week has been more than usually busy with stuff way beyond science in the lab, but it’s probably not necessary to enumerate. Suffice it to say that I certainly believe I am following the spirit of the Geek Manifesto, as well as fulfilling my roles around diversity work. I think that Philip saying that what I wrote about QFT was nebulous was a bit disingenuous. The challenge was not to write a 2 page Pathways to Impact statement as far as I could see, but to identify what would be the basis of one. I stand by what I said about areas and, if people want to know how to make a strong case they should read my comments here in conjunction with my the current post. If Philip worries that some practitioners in QFT shouldn’t be let loose in front of an audience of school-children, consider alternative ways of communication from writing a children’s book to a web-page. It isn’t difficult to be imaginative – if one tries. And that I think is the heart of the matter – are individuals willing to be imaginative? It seems to me that part of the problem about impact is that many people have clung on to its economic aspect to the exclusion of everything else, despite a lot of words being expressed on the subject. Richard has already expressed this eloquently when he said
It seems to me that opponents of the “impact agenda”, by interpreting it in the narrowest economic terms, immediately concede the most important ground, rather than taking the opportunity to say, yes, science does have an enormous impact, but it manifests itself in more complicated ways than some people think.
People may also like to look back at Mark Claydon-Smith’s comments on a separate earlier post; he is, you may recall, an employee of EPSRC. He said
I think your point about different style of reviewing is more about the nature of research quality. I always find it ironic that we can have endless debates about the categorisation of “impact”, when the elephant in the room is “excellence”. Quality is a subjective judgement about potential.
Given that research excellence remains, as EPSRC has recently reiterated, the over-riding criterion we should bear in mind that referees are actually the critical gate-keepers. And that means most of the people who are commenting on this blog. Referees are upon occasion highly subjective, possibly even biased (consciously or otherwise) about fields or individuals. As I made clear at the start of this post, personally my main gripe with the EPSRC has been about interdisciplinary research and Dave Delpy and colleagues have not infrequently pointed out that it suffers not least because referees are not good at refereeing such proposals (there are many other problems which aren’t relevant to this argument). So, can we move beyond the impact agenda and focus on how excellence is determined by our peers sometime for a change? How consistent is that across the board?
Philip keeps using the word ‘unethical’ and I am curious why he thinks that is appropriate. A quick look in the Shorter OED this morning gave these definitions of ethics (amongst others).
• Moral system of a particular writer or school of thought
• The rules of conduct recognized in certain limited departments of human life.
It seems to me that the rules EPSRC apply are both transparent and undoubtedly could be deemed to correspond to a certain school of thought. Just not Philip’s (and presumably other people’s). But inherently I don’t believe their actions are unethical so much as unpalatable to some. EPSRC’s goal about resubmissions had a clear aim – to reduce churn and the amount of work both they and the peer review system more broadly had to cope with. In itself that is a perfectly acceptable aim, and it isn’t that it is unethical, simply that the manifestation may have had consequences that not everyone likes. So, Philip, I would suggest you retain that rather strong word of ‘unethical’ for a more limited range of circumstances.
Finally, how do I respond to Michael’s comments, based on what Lesley Thompson said in public about ‘impact’. I can merely reiterate that this is not what I see happening in practice. The panels do have a lot of power to up or down the weight they attach to the impact statements; my experience is that it is not as far as I have seen the make or break criterion some people have been attaching to it. Lesley may have been making a point because there were Treasury officials present and this was the message she thought they wanted to hear, even if not what academics present hoped for. I don’t know since I wasn’t there. Since no one else has commented on their own experience of panels maybe my views are seen through rose-tinted glasses facilitated by my experiences with the BBSRC and possibly they happen to be more enlightened. I can’t tell. I can only say, in practice, I have not seen the sorts of things that are being attributed to the implied evil ways of the EPSRC. It is worth bearing in mind that many decisions – though arguably not enough – do reside in the hands of fellow scientists and that EPSRC Council has a number of practicing academic scientists on it too. In addition, that decisions are made by EPSRC staff has been true since way before the current Dave Delpy regime. For instance, around 2000 I applied for a Platform Grant and was told that the panel had left the final decision – to choose between two equally favoured grants for funding – to the Programme Manager. My application lost out. Did I go around sounding off about this decision? Not much, one just has to shrug one’s shoulders and move on.
”Given that research excellence remains, as EPSRC has recently reiterated, the over-riding criterion we should bear in mind that referees are actually the critical gate-keepers.”
Re research excellence: Of the eleven subthemes in its Mathematical Sciences programme, Mathematical Physics is the only one to be ranked excellent in international profile/standing by EPSRC itself. Yet in its recent Shaping Capability exercise it is the only subtheme to be reduced, in disregard of EPSRC’s own International Review of Mathematical Sciences.
Re referees as gatekeepers: only for those lucky enough to get past the programme-managers gate at EPSRC (Swindongate?). The rest were told to shrug and move on.
So, the obvious question this raises is “what’s the point of it?”
This is the cleft stick on which the impact devotees have impaled themselves: if it is being used as a tiebreaker between the too-many scientific cases ranked as excellent, then the tail is wagging the dog in that something that was supposed to be a minor factor is actually driving the process; and if it isn’t then why is it being included at all as yet another burden on overstretched researchers?
Because it’s important people think about these things, whether or not it’s a criterion. Those used to BBSRC ways will know that both data sharing statements and justification for use of animals are required. They may not be tie-breakers, but if someone can’t justify how many animals they need they probably haven’t thought their project through very well. The need to think about what impact is – in the broadest and not the narrowest sense of the word – is equally appropriate. And I have frequently heard it said at panels that those who can write an excellent science case tend to be the ones who can write excellent Pathways to Impact statements – because they’ve really thought about their work in the round.
I have no problem at all with that, and have said several times that I strongly support a mechanism that gets scientists to think about their research in a broader societal framework, including commercial potential, outreach, and simply the cultural value of what they do.
However, if that is the goal, surely the appropriate way to assess an impact statement is in a “pass/fail” mode *before* the science is ranked against its competitors. If that is made a formal part of the process, you would achieve the stated goal of getting everyone to think to an appropriate depth about these issues, but would remove the very real risk of tail-wagging-dog where it is used as the final arbiter rather than an an appropriate prerequisite for any science case.
Thanks Mike for spotting that particular pachyderm in the play-pen.
Athene, not only do you not really address Mike’s point but you missed two key issues I raised in the comment I posted in the wee small hours this morning:
– RCUK says that impact should inform the direction of the research. I am sorry to keep banging on about this but it remains unaddressed and, as I’ve said before (..a lot…), is right at the heart of the problem. How is public engagement meant to inform the direction of a research programme?
– Why do you think that researchers should be coerced into public engagement instead of EPSRC making available a separate (and substantial) pot of money for those who are enthusiastic about outreach? (And, by the way, Mike Merrifield – although he’s too modest to admit this – is one of those who is not only enthusiastic about outreach and public engagement but is extremely good at it. Take a look at some of the videos Mike has done for Sixty Symbols or Deep Sky Videos ).
In any case, one thing I find very unsettling is the idea that unless we’re forced to dream up pathways to impact statements we won’t get involved with public engagement. That’s a wholly depressing – and, more importantly, entirely inaccurate – attitude.
David Sweeney – of HEFCE, as you know but others who read this blog might not -noted in a public debate a couple of years ago that he was “shocked” to find that astronomers, particle physicists, and theoretical physicists (amongst others in physics!) spent so much time on public engagement.
Isn’t the fact that he was “shocked” rather telling! We’ve been doing this for years. Indeed, if anything, the REF and RCUK impact criteria will make me a great deal less inclined to get involved with outreach. I don’t do it to tick boxes.
Moreover, coercing people into particular activities simply to tick the right boxes so as to attract research funding can often have entirely counter-productive effects. (Mike M knows precisely where I’m coming from with this). I’ve mentioned Robert K. Merton before – he first put forward the law of unintended consequences…
You forgot to mention that as well as outreach I also commercialise my research (perfect Father’s Day gift by the way!).
But Philip is quite right: I didn’t do any of this because someone told me to, and I do find it rather insulting to suggest that I would only do these things because someone who considers themself a higher authority felt they should pressure me into doing so.
Athene,
I am not going to apologise for using the term unethical.
Any scheme which blacklists researchers on the basis of ranking at a peer review panel when we all know that this ranking is subject to a huge stochastic (and indeed sociological) element is beyond unplatable. It’s unethical.
I was invited to write a blog article for the Campaign for the Public University on Science for the Future. Because I really didn’t want to have to trawl through that lexicon of quotes you so love once more, and because last week’s events ignited debate in the ‘digital media’, I decided it would be more fitting to post a video .
Excuse the frankly sloppy editing but I hope it doesn’t distract too much from the content. ( Mike M: A certain talented video-maker of our acquaintenance will never want to work with me again if he sees some of the godawful edits. Let’s keep this amongst ourselves….)
Note that I again use the term unethical in this video.
A scheme whereby researchers are blacklisted on the basis of where grant proposals fall in panel rankings is far beyond unpalatable, Athene. It’s unethical. You know as well as I do that panel ranking is a far from perfect process. We can all point to cases where a grant proposal ranked near the bottom by one panel magically appears near the top when considered by another panel.
I will retract my use of the term unethical only when EPSRC carries out the simple experiment I describe in the video and shows me that there is 100% correlation in the rankings from panel to panel.
Acquaintenance ?!
Oops. Apologies.
Athene said:
“For instance, around 2000 I applied for a Platform Grant and was told that the panel had left the final decision – to choose between two equally favoured grants for funding – to the Programme Manager. My application lost out. Did I go around sounding off about this decision? Not much, one just has to shrug one’s shoulders and move on.”
Well, you might not be concerned. As a taxpayer, I am.
How did the Programme Manager make the decision? Was it a flip of a coin? If so, fair enough. But are the protocols for how (s)he made that decision well established and in the public domain?
Personal Reflections on Impact
My former boss, Steven Hill (head of the RCUK Strategy Unit) recently posted that science policy needs to take a historic perspective – although I am not sure what time period he meant! Many people have commented on the policies and philosophies associated with the RCUK/EPSRC Impact agenda; I have nothing personally to add to these here. Not many have reflected on the key personalities involved. Interestingly, most of the individuals involved in the major strategic decisions around impact are now running universities.
•Prof John O’Reilly – former CEO EPSRC and member of “Warry” group is now vice chancellor Cranfield University
•Prof Julia Goodfellow – former CEO BBSRC and member of “Warry” group is now vice chancellor Kent University
•Prof Keith O’Nions, who as DGRC commissioned “Warry” and initiated the whole impact agenda, is now Rector of Imperial College
•Prof Leszek Borysiewicz, as CEO for MRC oversaw methodological breakthroughs with Martin Buxton’s work on impact assessment for healthcare research, but more specifically coined the concept “pathways to impact”.
•Prof Philip Elser, former CEO of AHRC and RCUK Impact Champion is now Principal for St Margrets College London
•Prof Alan Thorpe, who as CEO NERC commissioned the first set of RC economic impact assessments, has moved out of higher education, back to meteorology.
•The impact agenda for RCUK was of course a forerunner for the new impact focused REF; the CEO for HEFCE that took this forward, was David Eastwood – now vice chancellor at Birmingham.
Most of the research council officials responsible for implementation have also left the scene, mostly through retirement, redundancy or resignation.
The “impact thing” is often portrayed as a political, Whitehall or even Swindon based invention – with a strong hint of “anonymous gray bureaucracy” to it. My experience is that this agenda was developed and led by key figures within the UK research policy, and these now have leading roles within our research intensive universities.
Mark Claydon-Smith
Between 2007 and 2009 (is) I was seconded to RCUK as Knowledge Transfer and Economic Impact Coordinator, working to Prof Philip Esler, then head of AHRC and RCUK Impact Champion. This is my personal perspective – I haven’t discussed with research council colleagues or management.
Reference – Warry report: http://www.vitae.ac.uk/cms/files/DTI-Warry-Report-July-2006.pdf
Mark,
Thank you for that intriguing post. Your comments are precisely in line with what I’ve long suspected.
The EPSRC argument, as you say, is always that the Council is responding to intense pressure from BIS, The Treasury, and government in general. I’ve long suspected that the dynamic is much more subtle than this: there are quite a few at EPSRC/RCUK who have invested a great deal of their reputation in many of the policy changes…
I would love to know just how many of EPSRC’s damaging policies have been introduced on the basis of ‘direct’ BIS/Treasury pressure and how many have been put in place because EPSRC was second-guessing what it thought government required.
Philip
I have answered your point repeatedly by implication: the reality is you interpret ‘inform’ as ‘direct’ and I don’t. If it all seems to hang on that one word but I have seen how ‘case law’ appears to work in practice and you haven’t, then I see no reason to believe your reading is appropriate. You don’t engage with that interpretation you just hark back to that one word. Since no one else has disagreed with me about what happens on panels (which may mean no one else who has read this far has sat on a panel) I will hold to my belief that operationally that is what is happening until someone posts a counter view.
Secondly, what happened to me around 2000 happened not because of the whim of an EPSRC employee but because the relevant panel ducked out of a difficult decision. Who sits on a panel? The likes of you and me, the scientists not the Swindon crew. I think the tax-payer might have something to say about the community itself in that case. I think that view holds too for what you call stochasity about where a proposal sits in the ranked list. The stochasity comes from the fact that some referees are remarkably cavalier, writing anything from twaddle to vitriol (with lot’s of good stuff in between) on referee’s forms. The whole point of my earlier post ‘some scores are more equal than others’ was that not all sub-disciplines even exhibit the same behaviour in the way they react to and score proposals. Of course there’s some stochasity (although in the case I cite I fear it was more like systemic bias) and that is not the fault of EPSRC but of those people who referee. It is also the fault of those people who decline to referee, removing their expertise from the pool, either explicitly (I’m too busy etc) or simply by not responding. The RC officials have to spend an enormous amount of time trying to get sufficient referees in order for a grant to get a fair hearing.
What would happen if resubmissions were allowed? There would be far more operational load, but also referees would end up re-refereeing proposals and I suspect that would only increase stochasity. Both because of additional overload leading to less time being spent on any given job, and also because it is easy to read a proposal you have seen once with jaded eyes if it turns up again. I try not to agree to referee a paper I’ve rejected once if it turns up on my desk from a second journal because I doubt I will be as objective as first time around, for instance. I suspect that attitude might turn up when referees get asked to referee a second time even if they are the most appropriate experts.
So, I suspect that many of the things that you find unpalatable are far more complex than simply being down to officialdom, and the scientific community has other responsible roles to play beyond simply getting cross with Swindon. Sometimes, we the community, overall do not act as responsibly as we could, such as around refereeing and panels. No one is perfect.
Finally Michael, I don’t remember anyone suggesting you only did the things you do because ‘someone told you to’ so much as since you do do these things you might as well write it down. But I don’t think it is unreasonable for individuals to expect to do something to satisfy a wider societal expectation. Again, if we are going to invoke this important person, the taxpayer, I reckon (s)he might feel this was reasonable.
Athene.
You’ll be relieved to know that this will be my last exchange with you in this thread. When we’ve got to the point where you’re invoking your superior knowledge of case law as an element of the argument in favour of impact criteria in peer review, I think it’s safe to say that we’re reached the nadir of the debate.
We’re scientists. We aim to express ourselves as clearly and as accurately as possible. (To the extent where all life is generally beaten out of the prose we write). I’ll not include the quote here (!) but I’m sure you know Feynman’s words on the importance of clarity in the writings of scientists.
We should expect the same clarity of expression from the research councils.
Your argument is that each panel should interpret impact as it sees fit (or, more accurately, as you see fit). If you cannot see the glaring inconsistencies in that approach then I’m not going to start pointing them out.
But thank you for finally addressing my dogged (and no doubt irritating) pursuit of clarification of the term “inform”. You confirm what I said right at the top of this thread in response to Richard (Jones): “What you and Athene believe EPSRC/RCUK says about impact and what it actually says (and requests) are two very different things.” .
Moving on to your points re. the vagaries of peer review. In order to counter my argument about the ‘stochasticity’ of peer review and the unethical nature of the blacklisting process you state that EPSRC can’t be held responsible because referees are human. And that all panels behave differently. And that, therefore, peer review is stochastic.
That’s not counter-argument. It’s tautology.
You haven’t begun to address my point. To reiterate: Blacklisting is inherently unethical because it is based on a process which has an inherently stochastic component (as you yourself admit). Just because a grant proposal falls in the lower half of the panel ranking does not mean that it’s necessarily a poor quality application. And, by extension, just because a researcher has had three proposals fall in the lower half of ranking at panel does not necessarily mean that the researcher in question is incapable of writing high quality proposals and therefore needs to be mentored. And that a letter should be sent to their Head of School/University pointing out that the quality of their proposals is not up to scratch.
On resubmissions: Let’s just say that I don’t share your in-built bias against resubmitted papers or grant applications. In my opinion, one of the people best placed to ascertain whether authors have dealt with the criticisms of a paper/proposal is the reviewer who made those criticisms in the first place. My confidence in the scientific method is always made that little bit stronger when we submit a paper that gets a hard time from a referee, we make changes and/or do more experiments/calculations, and eventually win over that referee. Similarly, I have been deeply sceptical about some papers I’ve had to review which were improved immensely after revision.
I attended a day-long workshop yesterday which EPSRC organised for those who currently hold EPSRC fellowships. Despite my initial reservations, this was an important and very useful day. The issues of the loss of project studentships, the fallacy of introducing a consideration of impact at the proposal stage, and the worrying lack of transparency in some aspects of EPSRC’s review processes were raised. (And not just by me. Indeed, I was relatively subdued. It was immensely encouraging to see a number of the concerns highlighted by SftF raised at the meeting yesterday by those who EPSRC considers ‘advocates’ for UK science and the Council itself).
I’ll leave you with a question you didn’t address in my previous comment and which I put to Atti Emecz and Andrew Bourne yesterday:
“Why do you think that researchers should be coerced into public engagement [via Pathways to Impact] instead of EPSRC making available a separate (and substantial) pot of money for those who are enthusiastic about outreach?”
I have decided to become a little more active in engaging in these blog debates and will do so as long as Athene is content to host an on-going discussion.
There have been a range of issues raised in this blog and I could not hope to pick up on all of them in one post. I will therefore restrict myself to responding to two of Philip’s points.
Philip states: “I find it just a little disconcerting that Athene and Richard (and, more importantly, EPSRC itself) discuss grant awards in terms of the Council awarding the funding. EPSRC shouldn’t be responsible for selecting who is awarded funding. It should be there simply to disburse the funding.”
Our position is pretty simple. Council has to award the funding. Council is fully accountable for the use of the public funds entrusted to it and this includes delivering a programme of activities agreed with government as set out in our Delivery Plan. If funds are not used to best effect, responsibility for that must lie in the Council and its executive and not in any advisory structures. That statement is not meant in any way to downplay the importance of such advice or our advisors. I would emphasise also that the Council is EPSRC. The authority that I may have as a member of the executive is given to me by Council and exists only to the extent that Council agree to give it.
In another post (May 22, 2012 at 2:24 pm) Philip requests evidence that directed research produces greater levels of innovation than exploratory research. I want to point to some evidence which I hope is helpful to the reader even if that evidence answers slightly different questions. The research ecosystem is, of course, complex, dynamic and interactive. It is not easy to control individual variables and I think that evidence is more likely to be in the form of indicators rather than hard proofs. Anyway, I would point to the following as evidence/indication that directed research is of similar quality to exploratory research.
EPSRC commissioned Evidence Ltd to look at our portfolio of research back in 2008 – the full report is here. http://www.epsrc.ac.uk/SiteCollectionDocuments/Publications/Other/citationstudy2009.pdf This covered research across all our portfolio (including physical sciences). It concludes: “There is no evidence for any significant difference in citation performance for papers arising from ‘Responsive’ and ‘’Targeted’ funding modes.”
NERC did a similar study http://www.nerc.ac.uk/about/perform/documents/citations-study-2008.pdf . Two conclusions from that: more papers come from core strategic funding than other modes and most citation impact comes from their fellowships
Moving on, EPSRC regularly reviews the way it works and we publish these reviews – some recent ones are here http://www.epsrc.ac.uk/newsevents/pubs/reports/Pages/schemeevaluations.aspx This includes a review of platform grants (which some have argued are not responsive or exploratory although we in EPSRC would disagree) and IRCs. These reviews provide evidence of the added value of these approaches (and hence might begin to provide the sort of evidence Philip was after).
I might try and dig out some older reviews – I was involved in the Magnetism and Magnetic Materials directed programme in 1990-1993 and I now feel a nostalgic urge to find the review SERC/EPSRC did on that. From memory, it was pretty positive both in terms of research quality, linkage to industry and in building a community.
Finally, as I think Richard Jones pointed out, EPSRC (Council and executive) is always open to receiving additional evidence.
David Bott has drawn my attention to his own contribution to this debate on the TSB blog here. He sums it up as “this is the science that has made money, and this is the science that hasn’t made money – YET!”
… thereby succinctly summarising everything that is wrong with the impact agenda, by marginalising all the excellent culturally-enriching science that will never make any money for anyone.
Philip
This is my last comment here, to avoid blog ping-pong.
I think you have misinterpreted or misunderstood what I was saying with “… quite a few at EPSRC/RCUK who have invested a great deal of their reputation in many of the policy changes” – i.e. I was trying to say the opposite; few of the seminal “impact” policy makers are actually still around as vested interests. Like a West End production, the lead actors have left (mainly to run universities), the supporting crew turned over, but the show rolls on.
In my personal opinion, there have been no major research council policy developments in impact in recent years – the closest is probably the AHRC Connected Communities/Big Society initiative, although I would argue this is just a slightly different manifestation of “challenge led innovation”. The more significant and interesting policy developments have all been contextual – i.e. abolition of RDAs, broadening role of the TSB, development of the Catapult Centres and the emergence of a political narrative around “industrial policy”. These do all impact (excuse the pun) on the research councils – as well as the universities obviously.
Again, my own opinion is that the current controversy has not been caused by the “impact agenda” as such. Rather the heart of the debate is how to “maintain excellence, in a time of increasing cost and reducing budgets”. EPSRC Council has developed its “shaping capability” approach; some have challenged this based on a range of why, what, how and who issues. Whatever happens, we need a strong argument for science going into the next spending review – supported by a clear and settled strategy.
Mark
ps in identifying the key policy makers involved in impact I overlooked Prof Ian Diamond, who as Chair of RCUK Executive Group oversaw much of the RCUK policy development over this period. Ian is now Principal and Vice Chancellor for Aberdeen University.
Mark.
I take your point re. current vs previous EPSRC management. But the key message I took from your comment was that the oft-made claim that EPSRC/RCUK decisions are driven by BIS/Treasury pressure is far from the whole story. Atti Emecz, EPSRC’s Director of Communications (see posts above) stated during a meeting yesterday that the dynamic is indeed significantly more subtle than it is usually painted.
Best wishes,
Philip
Hi, Atti.
Good to speak and debate with you yesterday. I think it’s great – and to your, and EPSRC’s, immense credit – that you are also engaging via media such as Athene’s blog and YouTube.
You respond to two particular criticisms I have made.
1. The first is on the extent to which EPSRC staff, rather than peer reviewers and panel members, should decide on which grant applications to fund. Apologies if I was not intially entirely clear that this was the point I was making. I entirely understand and appreciate your argument that Council is ultimately responsible for disbrusing funds to the best of its ability.
But, as was noted on a couple of occasions during the “Developing Leaders” meeting yesterday, there is a worrying move in EPSRC towards awarding funds via far from transparent mechanisms. In answering one of the questions from the floor (I don’t think it was mine in this case!), you said that “There are different mechanisms of peer review” and that what’s appropriate for some schemes might not be appropriate for others.
I must admit that this didn’t fill me with confidence.
As I’ve mentioned to you before, why should fellows get preferential treatment when it comes to funding? If we’re good enough to have been awarded funding for a fellowship then surely we should be more than capable of competing against our peers in open competition via standard grant application procedures? I find it rather embarrasing that the “New Directions” scheme bypassed the grant application and peer review process which my colleagues who are not currently fellows have to undertake.
I take on board your point about extensions in other (non-RCUK) fellowship schemes. But five years of substantial funding plus a reduced teaching load should be more than enough to ensure that a fellow can compete very well on the same ‘playing field’ as everyone else. We shouldn’t expect also to have preferential treatment via far less robust peer review mechanisms.
2. You also address the issue of evidence. Importantly, you include the proviso: “The research ecosystem is, of course, complex, dynamic and interactive. It is not easy to control individual variables and I think that evidence is more likely to be in the form of indicators rather than hard proofs.”
I agree entirely. The point I was trying to make above – albeit, in a somewhat convoluted fashion – is that it’s simply not possible to arrive at a figure for the total socioeconomic impact of basic research. A number of important studies, including that Salter and Martin paper from 2000 to which I refer above (and with which you’re as familiar as I am!), make this point very clearly. Getting credible evidence on the total economic impact of basic science is practically impossible.
You highlighted yesterday that despite the frequent calls by government for evidence, in many cases it is case studies and compelling stories that best make the point. As Philipp Kukura urged, EPSRC should put its efforts into developing these case studies – i.e. highlight impact stemming from research which has been funded – rather than devoting so much of its time – and reviewers’/applicants’ time – on the Pathways to Impact process.
Both Polly Arnold and Philipp Kukura raised points which, from my discussions with other fellows yesterday, were widely supported. I would very much hope that their comments will, ahem, inform future policy development at EPSRC.
You quote the following: ““There is no evidence for any significant difference in citation performance for papers arising from ‘Responsive’ and ‘’Targeted’ funding modes.”
But this could be interpreted as an argument against targeted funding? Why invest all that time and effort into managing and administering targetted programmes when they give you no benefit above what you get from responsive mode?!
Hi Philip
I thought it was a great event on Monday and I am pleased you found it helpful. I enjoy such events and it is always such a pelasure to hear from our fellows (even when they disagree with some of our policies) as they are our future (or even existing) research leaders. Thanks for your kind comments as well about our willingness to engage. I will try (but probably fail) to keep my comments relatively focused here and suggest that further discussion should be undertaken away from Athene’s blog. My thanks again to her for hosting this to date.
1. On peer review. For the record, please be assured that all our grant funding decisions involve peer review. Sometimes we use algorithms (e.g. for the Doctoral Training Grant) but the algorithms are always based on peer review outcomes. Council recently discussed this and re-affirmed its commitment to peer review. You are aware we also publicly (in Nature) committed our commitment to excellence as primary criterion.
I am sorry you still feel uncomfortable about different forms of peer review. I maintain that this is correct (and good!) and hope to give you some assurance by stating that that over my 20 years in Research Councils we have always had tailored forms of peer review to meet different needs e.g sometimes there is an outline stage, sometime no postal review, sometimes face-to-face interviews.
2. I entirely agree we need to work up case studies based on past successes together and I look forward to doing so with you and others ahead of the next Spending Review. We will probably need to work up some evidence to support specific points but again would agree we will not have a single number about value of basic research.
I was making the point about research quality in responsive and targeted modes because we are often questioned on that. Your response seems to suggest that we use managed programmes just to improve research quality. While we are always looking for ways to do that, managed/targeted programmes are often set up with other objectives in mind e.g. building up capacity, stimulating inter-disciplinarity etc so I would argue their value is broader than your point implies and they must be judged against that.
I will stop now in case we end up agreeing on too much 😉
Readers may be interested to know that Adam Smith has written an overview of some of this debate over on the Guardian here. There the debate has turned more to EU funding, but I am slightly curious to see that Philip has introduced a new dimension. It appears that he is perfectly prepared to go in for gamesmanship/spin on EU grants saying:
There’s a very simple game to play with FP7 applications. (Again, I want to stress I’m not talking about the ERC here). You take the advice for the referees and make sure that all the buzzwords that are highlighted in bold in the guidance document are reproduced (frequently) in your proposal.
Intellectually bereft and soul-destroying to write.
But it ticks all the boxes…. ker-ching.
Clearly he sees this as something different from the science being eroded, as he claims EPSRC policy imposes. But I’m not convinced by the purity of the difference. But since he’s recently (somewhere in the comment stream above) accused me of reaching the ‘nadir of debate’ and didn’t choose to engage with the idea that some of the perceived EPSRC problems lie with how we, the community, behave when in referee/panel mode, it is probably time to stop.
So, your argument is now that just because FP7 funding has no good basis in science, it is somehow more acceptable for EPSRC to move in that direction, too?
Surely, the fundamental difference is that since UK research councils are answereable only to the UK, we might credibly hope to align them with what we perceive to be in the UK’s best interests.
Michael
I was passing no comment on FP7 funding, about which I know very little. All I was saying was that I was surprised to see Philip’s attitude towards game-playing with the EU, given his resolute stance with respect to the requirements of the EPSRC.
I buy lottery tickets. It doesn’t mean that I want UK research councils to allocate their funding any more randomly than they already do.
FP7 funding is fundamentally different from UK research council funding, so, not surprisingly, people reach different conclusions about whether they would rather spend their time playing the game, or spend it trying to get the rules changed to something less objectionable.
Athene,
Re. the ‘nadir’ of the debate:: Your argument was that as I know less of case law than you did (or – more correctly – that you perceived I knew less of case law than you did), I was not sufficiently qualified to interpret the EPSRC/RCUK position on impact.
If, as scientists, we’re reduced to attempting to ‘trump’ each others’ arguments on the basis of case law then what is the point in continuing to debate? Has science funding policy in the UK really sunk that low?
For completeness, however, let me address the ‘conundrum’ of why I apply to the EU for funding but not to the EPSRC.
– The reason I no longer submit EPSRC grants stems directly for a call I made with a number of others back in 2009 (via the Times Higher – the link is in one of the comments above) for a boycott of review of the “Pathways to Impact” nonsense required for EPSRC proposals.
– I told EPSRC I was happy to review all aspects of a proposal other than the Pathways to Impact statement. They said this wasn’t acceptable. So I resigned from the peer review college.
– Having resigned from the peer review college, it would be unconscionable to submit proposals, wouldn’t it?
– Not even the Commission (via its FP& funding schemes) asks for impact “to inform the design of your research”. Not even the Commission argues that you should attempt to predict the importance of your research on a 50 year horizon. (Coincidentally, as is clear from another comment posted under Adam Smith’s article, my interpretation of the verb “inform”, far from being a “misreading”, coincides very nicely with EPSRC’s understanding of the word. It is also entirely in line with David Delpy’s statements regarding a culture change within UK academia. Feel free to rewrite the OED with RCUK’s help, if you like. Just let the rest of us know when you do so…)
– The Commission fundamentally recognises the value of blue skies/fundamental research via the European Research Council. (Have you listened to Helga Nowotny (President of the ERC) speak on the value of basic research? Compare and contrast with the corporate, jargon-ridden public pronouncements of our national research council representatives.).
Very happy to expand on any of the points above if necessary.