Having just returned the grant review session at the NIH I thought that this would be a particularly good time to bring up something that has been bothering me for a number of years.
It’s quite sad, but I’ve slowly come to the opinion that in science, just like in economics, money is “the mother of all metrics”. When I was a student, money mattered because without it one could not buy antibodies and other essential reagents. But when it came to comparing which laboratory did better research and made more important discoveries, the measure of all things was always the publication of papers.
As a postdoc at the NIH, money was rarely considered–perhaps because it never was an obstacle for anything. Laboratories received slots for postdoctoral positions based on research productivity. Each postdoctoral position came with the sum of money allotted to the lab for reagents and equipment. So it was only natural to compare the science of one lab with that of another by looking at their publications––as an indication of scientific achievements.
When assuming my position as a principal investigator some years ago, I remember the shock I felt when I attended my first seminar in the new department and heard the introduction of an external speaker. Much praise was heaped on the speaker for the number of grants he held and the number of years he had held them for. On the other hand, relatively little was said about his publication record and most importantly, the seminal findings made by this researcher.
I’ve been talking a lot with one of my senior graduate students recently about options for choosing a postdoctoral laboratory. In the course of these discussions, I realized that an important criteria might be the ability to sum up a researchers life work by several bullet points: in other words, if the researcher is truly been not just productive, but has also managed to progress in some linear fashion within his or her given field, this seemed to me to be of particular importance.
Of course, the researcher needs funding in order to buy reagents and carry out research, but surprisingly there seems to be wide general agreement among many colleagues that I have spoken to that there is some magical number of personnel beyond which a researcher loses his/her optimal per capita output. For example, laboratories that have up to eight researchers-including postdocs, students, and technicians-seem to have more publications per researcher then laboratories that are larger than that. This of course raises the issue as to whether it is wise for funding agencies to continue to heap multiple grants and large sums of money on laboratories that are already well-funded and have huge numbers of people working in them.
Most of us are probably familiar with some of these super-laboratories, where if the number of people working there are disregarded, the output of the PI is phenomenal. For example in a given year such a PI might publish a Cell paper, a Nature paper and a Science paper. These really are outstanding accomplishments. However, if one follows this researcher’s output over time and realizes that there are 19 or 20 postdocs in the lab, and that only 3 to 5 of them are the ones actually publishing these outstanding papers, that leaves a lot of unsuccessful and frustrated postdoctoral fellows behind. Of course, one could argue that in a capitalist type of system, this is the way things work; the top postdocs succeed, and the others, who are either less capable or less lucky, sink.
So when I advise one of my own students or a student from my department who comes to me asking about advice for postdoctoral positions, I often counsel them to take these kinds of issues into consideration. Yes, some of these labs are phenomenal places to train, and present outstanding opportunities for highly motivated, self-sufficient, and yes, lucky postdocs who manage to be in the right place at the right time. However, they do need to take into consideration that if they fall in the category of the 75% whose projects are not headed for Cell, Science, or Nature papers, they need to be aware that they can end up in a difficult situation.
So overall, while money may make the world go round (and grant funding may dazzle the universities), I think it’s critical for us as scientists to remember that money is not the measure of all things. For when we retire or expire, we will not be remembered as the scientist who had this or that much funding, but as the scientist who advanced a certain field by his or her specific findings. I’ll take findings over fundings any day.
Steve, I’m with your final conclusion definitely. I also agree that labs with huge numbers of people -20 and upwards, which is not uncommon in some fields such as synthetic chemistry – do not necessarily provide a very stimulating environment or one that is good for training. There is little mentoring (only perhaps from senior postdocs) and the lab may resemble an assembly line, with each student merely playing a bit part in the whole scheme of things, rather than obtaining a well-rounded education.
Thanks, Athene, I’m glad to know that you agree!
I think you are 100 percent right. It is much the same in the UK. There is a disastrous culture of big science, in areas chosen by people who are not scientists, but research council staff. Smallish responsive mode grants have almost vanished, despite the fact that they are widely believed to give best value for money.
I have known four Nobel prizewinners quite well, and worked with two if them. All four had small labs and all four did experiments themselves up to quite an advanced age. Their modus operandi was precisely the opposite of what research funders are now pushing towards, If they go on like this, don’t expect many more Nobels.
I’d be very interested if anyone has hard numbers for productivity vs lab size. OK I’m quite aware that you can’t measure value by impact factors or citations, but something is needed to persuade funding agencies that they are corrupting good science (email me at d,[email protected])
David,
I have no hard data or real statistics to back up these assertions, but it would definitely be good if there were a way to compile statistics to look at this. Science seems to be removing the individual from having an opportunity to come up with his/her own independent ideas.
Steve – I’m sure that what you say will resonate with other scientists, but your final assertion “that money is not the measure of all things” is simply not the view of University managers and administrators. These are the people who decide on promotions and appointments, and to them the university is a business and money trumps everything. So while Cell and Nature papers may win the approbation of your peers, they cut little ice with your ‘superiors’. I have seen the consequences of this in action (no names for obvious reasons).
In one high profile University department there was an investigator who had an extraordinary knack for raising funds, and I’m talking about many millions from wealthy donors, rather than through peer-reviewed grant applications. Thanks to the vast overheads that came with these funds, this individual brought financial security to the entire department which benefited all, yet this scientist rarely published anything, and certainly nothing of note. This investigator had a post-doc in his lab, an unremarkable individual who wouldn’t have stood a hope of a faculty position in any open competition. Yet in this department, that post-doc had a coveted faculty position custom-designed for him/her, essentially as a thankyou to their money machine PI.
Other post-docs in that department (and elsewhere) with much better publication records were effectively denied the chance to compete for that faculty post, solely because their scientific contributions were judged to be worth less than hard cash.
Stephen,
I agree with you completely about the administrators and universities seeing it all in business terms. There is no question that this is true, and your example is one that I have seen and heard in various versions. Nonetheless, I think that we scientists (those of us who idealistically still believe in the finding over funding) should do our best to offset this trend by awarding the real respect to those who not just obtain funding, but do great scientific work with it…
Stephen,
Your story is so familiar that I can’t help wondering if we’re thinking of the same PI…
Or, its a widespread problem.
Very interesting topic. Yes, universities and research institutes are run as businesses these days, and grant money/industry money/donor money speak much louder than publications and hard data.
I’m struck that eight is often pegged as the magic number of lab employees. My own current PI used to say “After eight, the wheels fall off”, and apparently once swore that he would not grow his lab beyond eight. But he has grant money, and people (including your truly) keep knocking at his door begging to come on board. And he finds it hard to turn away promising candidates when he has the money. As a consequence, we’re now way past 8 lab members. I can see him struggling to keep up with mentoring everyone’s projects these days (esp as he climbs the ladder and admin duties increase), but I guess it’s hard to resist expansion when the possiblity opens…
The number 8 was only a guesstimate, and I am sure that there are labs with more people that are still highly productive on a per-person basis (as well as smaller labs that are less productive). But I think that unless there is some real delegation of authority with top-notch underlings directing at least some of the research, once you get above 10-12 people I suspect that there will always be a number of people who get left behind, even if they are motivated and talented…
What’s more, these days, it costs £££ to publish papers. Especially if you want open access.
(Runs away)