How often have I heard it said that ‘the policies are good but the implementation is shaky’? I could make that comment about many of the issues around women in science, where the best-intentioned policies are defeated by negativity, implicit stereotyping and just plain old habits dying hard and not being challenged in some remote parts of an organisation, whatever the top brass may believe. However, in this post I will leave the diversity theme alone and return to the topic of a relatively recent post, that of interdisciplinary work, where I believe the same statement about the mismatch between policies and implementation may sometimes apply.
Previously I wrote about the challenges facing those researchers whose area of activity cannot neatly be pigeonholed into mainstream areas or whose work spans more than one Research Council’s activities. The post was prompted by a meeting both myself and Doug Kell, Chief Executive of the BBSRC, had attended – Physics Meets Biology. Both of us had come away from the meeting enthused and encouraged by the exciting research being carried out in many places within the UK. However, I retain a long held worry that such interdisciplinary research fares badly under the current grant-awarding system where I suspect there is an overwhelming tendency of ‘regression towards the mean’. In other words, I believe truly innovative and adventurous research that doesn’t tick standard boxes can fall by the wayside.
My experiences with both the BBSRC and the EPSRC over many years have fuelled this anxiety. The situation can only be made worse by the current funding climate, where money is exceedingly tight; it is perhaps unsurprising if, in these circumstances, panels tend to play safe and the more adventurous/less mainstream/ less easily pigeon-holed applications are systematically if unconsciously disadvantaged. (Yes, alert readers of my blog will spot that here is a very different version of unconscious bias than the one I usually write about!)
Doug attempted to refute what I said on his own blog. He highlighted what the BBSRC policies are around interdisciplinary work, and also on his blog discussed the (un)importance of hypotheses, an issue I highlighted in my earlier post as causing particular problems for scientists like me when applying to BBSRC. Although, like him, I do not think the blogosphere is a very good medium for this debate, I would like to think that our exchanges could prompt much deeper thought within the Research Councils and the wider community. So, I will take the liberty of analysing his refutation. It should be recognized that this is not meant to be read as an issue specific to BBSRC (whom I am sure are no worse, if not any better than any other Research Council), let alone directed at Doug Kell personally.
The basis of my concern is that, whereas Doug may be absolutely correct in what he claims is his Council’s policy, there can be – and I believe indeed is – a gulf between excellent policies and the actual implementation. We all know that the road to hell is paved with good intentions and I would contend that, whatever the good intentions and stated objectives of BBSRC (and probably EPSRC too, although they haven’t entered this debate, at least as yet here), their implementation is not monitored to see where these good intentions have actually led. This is a common problem for large organisations with complex structures and many layers where decision-making may occur. I have no reason to believe the BBSRC is any different from any other organisation in this regard, but if – as BBSRC and EPSRC explicitly state – they are both truly committed to funding interdisciplinary work, they should check whether their well-intentioned policies really are delivering what they want or whether somewhere along the route systemic problems arise.
First of all there is the question of which Research Council to send a proposal to when it straddles the seams. Doug says that if researchers are worried where their proposal might sit they should discuss it with staff. I cannot tell you how fed up I am with this answer, because I have heard it repeatedly and it completely misses the point. The remit query (set up on the Council’s website), and conversations with helpful staff in Swindon, assume that a grant can be written so it fits readily within one Research Council’s ambit or another. I am not interested in how the funding is carved up and I am well aware two or more Research Councils may co-fund a grant in some proportion or another. I am interested in whether the application that describes the best science an applicant can write up will be judged fairly by committees. I have been in the position where my grant has been rejected by (in this case) EPSRC because there was too much biology in it, but there was no way it could find favour with BBSRC (who were prepared to accept it) because the biology was necessarily of the Noddy kind which would allow the physics to be teased out on a model system before there could be any possibility of tackling the real biology.
We are always told by those kind Swindon folk, to write the best science we can. Then we are told it isn’t acceptable to one Council and it is very obvious it will be bound to fail at the other. When this happened to me, and I was told I had to withdraw my EPSRC application and submit it to BBSRC, I was well placed to judge just how well the grant would fare at BBSRC, because I was chairing the relevant committee at the time! The ‘best’ science may well have a lot of rather simple biology in order to progress an approach before it can be used on serious biology; in order to achieve this the physics needed may equally appear to a hardcore physicist to be Noddy. The synergy is what makes it interesting and worth doing. Being forced to find a home in one Research Council or another will merely destroy that synergy. I assume these same arguments apply at all the junctions between Research Councils. It would be good to see comments from people who recognize the problem at any of these joins; I saw a smattering on Twitter after my last post on this topic.
Let me now turn to the vexed question of hypotheses. Of course Doug will know how the referee form reads and he tells me the current wording is ‘If the work is proposing or testing hypotheses….’, but I wonder when he last sat through a grant-awarding panel or read a batch of referee reports.
How about this as a response from one referee to my non-hypothesis led proposal to BBSRC from a couple of years back.
But as hypothesis‐based research, this proposal is rather weak and unfocussed. In fact, the word “hypothesis” never appears in the Case for Support. Moreover, the implicit hypotheses are not compelling. The proposal has the flavour of a “fishing expedition”, in which the investigators intend to play around to see what their technologies can deliver.
This referee then proceeded to try and invent some hypotheses (s)he then shot down. What a way to proceed! Even if a committee and its chair try to ignore such stupid refereeing, the damage is done. When stacked up against other proposals which have only positive referees’ reports of course it will be subtly marked down. It doesn’t matter that the proposal was not intended to be hypothesis-driven, nor that the BBSRC senior management doesn’t want grants to be judged this way, what I refer to as the tribal nature of the community defeats the good intentions that the policies may intend. So what is the answer? I am afraid that just being told that of course it isn’t necessary to include hypotheses simply doesn’t solve anything.
That is why I would urge the Research Councils to look again, not at what they say they do, but how their policies actually work in practice. To be told that the Chief Executive had no trouble getting funding for interdisciplinary when he was an active researcher also isn’t reassuring; once upon a time I didn’t have issues either, but those days are past. I think the difficulties for interdisciplinary work are now acute and only those areas (eg EPSRC’s Technologies for Healthcare) where a major, genuinely interdisciplinary programme is formally constructed with money explicitly allocated to it, is success likely. For those of us who work in more marginal or ill-defined areas which cross boundaries, but not as part of a pre-determined priority, life is an endless frustrating struggle, with success having little to do with excellence.
So I say again, maybe it is time for those of us who do not fit squarely into any box to stand up and be (even more) vociferous. Not to use special pleading for our field to be particularly protected, but to call for practices to be as fair to us who work at the margins of any given discipline and who cross boundaries, as for those who sit squarely in the middle of some well-established subject. I would urge the research councils to look at, not their policies, but what actually happens in practice so that they can give me the evidence that interdisciplinary work is genuinely not being disadvantaged by current structures. It is of course possible my experiences are not aligned with other people’s, but nothing that has been said to me by colleagues makes me think that is so. Again, comments on this blog – one way or the other – might help to beef up the evidence. The plural of anecdote is not data, but a multiplicity of examples could help.
Once again a very interesting post. One of the issues here, and this may apply more to EPSRC than BBSRC in terms of the differences in their respective review processes, is that “we are the research council”. I use the quotations here as this was stated by a Chief Executive of a research council recently at a meeting I attended. The reason why this was said was because someone at the meeting was objecting to the fact that their interdisciplinary research had recently been rejected as it “fell between the gaps” of two research councils. The response given was that the research councils do not reject grants, it is more the referees and panels, consisting of members of the community, that “reject” proposals. Herein lies the problem. The research councils have a policy to support interdisciplinary research, but the community at large can consist of often factions and not-so liberal minded individuals who fail to see the often important blurring of lines between disciplines – where physics and biology meet is not as well-defined as the separate disciplines themselves. It is challenging to work in this domain etc. So, it is they that misinterpret the research grant that comes for review, only seeing it from the approach of their own discipline. I’m not sure then that it is always the research council’s fault, and perhaps more has to be done by us to educate the community on the importance and the nature of truly interdisciplinary research.
Steve – I know the EPSRC say it is the community that does the refereeing and therefore the community that is at fault. I discussed this response in my earlier post . I don’t think it entirely stands up because the EPSRC staff choose the referees and do not have a pool – as Dave Delpy had once suggested – of those who specialise in interdisciplinary work or who are known to be good at doing that task. So, if staff send proposals to those wedded to a more single-minded way of doing science it will necessarily bias the refereeing process.
My Hypothesis is that this problem is an inevitable result of the professionalization of science.
In the process of professionalization, certain practices and beliefs become canonized -these then become the standards that protect professionals from charges of unprofessional conduct. Education is then a ritual that promotes these standards in order to create an epistemic community.
So, by definition, professional standards cannot condone research that is outside professional canons. To do so would be unprofessional and to relax standards would undermine the very nature of professionalism.
This suggests that historically, most “revolutionary” research must have taken place before professionalism was fully in place -or that such research must increasingly be done outside the professional (academic) framework. Einstein’s revolutionary work, for example, was done while he was a clerk in the patents office -so in that sense, he was still an amateur scientist. Would one get funding for similar innovatory work today?
Nowadays, academic funding and career patterns seem to exclude working with amateurs -but collaboration with commercial (and military?) partners appears to be very acceptable. In this
context, one should note that amateurs plus (some innovatory) commercial systems and the military are probably most free to put their resources where their mouths are -in order to think outside the box: Supporting research that may be outside the existing canons of the professional academic system. But should commercial and military systems be given the monopoly on innovative research?
Apparently, the historical rise of the commercially funded applied scientist during the industrial revolution caused a split with the more “aristocratic” pure scientist. Resulting in a rise in professionalism but a separation of creative potential. For a while, the “amateurs” working within a “gentlemanly” academic system were presumably able to keep the system going -but the rise of rational (and tautological) funding systems makes innovation increasingly difficult.
If the rise of professionalism does canonize standards that prevent the very innovation that one might expect professionals to achieve -then this could be a manifestation of Goedel’s paradox.
As a practical test, it might be interesting to investigate the effect of US ARPA funding of Project Mac on the British computing industry and its previously innovatory academic computer scientists (in Manchester, for example): I believe this effect was correctly predicted by Sir Leon Bagrit in his 1964 BBC Reith lecture.
Strange that we have the means to create an “aristocratic” society based on the economic exploitation of machine slaves -but we have apparently chosen instead to put humans in direct economic competition with the machine: Thus creating a social and cultural destruction rather similar to the barbaric slaughter in the trenches of WWI. If a system is moving in a dangerous direction, then the more rational and efficient it is, the sooner it will self-destruct.
“Italian scientists jailed for ‘false assurances’ before earthquake”
guardian.co.uk, Monday 22 October 2012 19.27 BST
Luciano Maiani, the incumbent president of the Major Risks Commission, said the verdict marked “the death of the services provided to the state by professors and professionals. It is impossible to supply the state with advice in a professional and composed way under this crazy judicial and media pressure. This does not happen in any other country in the world.”
Giampaolo Giuliani, the researcher who became the “Cassandra” of the disaster after his warnings were ignored, said he had expected lighter sentences.
But isn’t the certainty implicit in responsible governmental advice completely opposed to the uncertainty implicit in responsible science?
This comment has been sent to me to add in anonymously (for obvious reasons), but addresses a different aspect of the challenges of interdisciplinarity
From Anonymous
EPSRC’s recent shaping capability exercise, in which they set priorities for investment, is another way in which they have tried in a well-meaning way to address interdisciplinarity. In ICT they had a special call in the summer entitled “Working together”, for proposals combining ICT different specialisms, or ICT with other disciplines, and “Working together” is also one of five priority themes for Fellowships.
This is in one way to be welcomed, as unlike some of EPSRC’s other fields, where EPSRC restricted to Fellowships to a small number of areas, this means that no-one is excluded from applying.
However word on the street is that EPSRC are now finding it very hard to find referees for “Working together” bids, and with the panel meeting now quite close, applicants who ask EPSRC when they will get their referees reports to comment on get a rather sheepish reply.
I hope that when the process is over they do a post-mortem to see if the call has achieved what they wanted. Applicants were meant to address the following questions in their bids – taking up valuable page count for generic woffle at the expense of the science.
• What will be the benefit of the research streams running concurrently?
• Why is it important to have a project bringing researchers in these areas together?
• Why isn’t this happening already?
• How will this result in a different ICT research portfolio?
• What will they do to ensure this project contributes to greater working together amongst ICT researchers?
• How do they think this will effect ICT research in the UK in the beyond the boundaries and end of the project?
I hope that the whole process will fail -simply because the iCT paradigm is a disastrous one.
In the early days, computing might have gone under the label of “Process Technology”. This would have been more appropriate -because “information” then becomes understandable in the context of Process -while an understanding of Process tends to get blocked in the context of “information”.
The classic definition of “Information”, from anthropologist Gregory Bateson, was: “The difference that makes a difference”. This clearly places it in the context of a state in an (unspecified) system that then acts as a “switch” which causes a change in some other (undefined) process.
The importance of such a definition lies in the undefined nature of both systems involved -because this reveals the subjective nature of information. However, ICT treats “information” as being something that has intrinsic “objective” value -and therefore (consciously or unconsciously) imposes an entire system of pre-determined (and indoctrinating) interpretations upon the recipient. The age of information -is fundamentally the age of indoctrination. However, I severely doubt that any proposed fellowship would be likely to uncover such things -because the interpretational framework of ICT has already been fixed (exactly as in a “fixed” football match).