How many times have we been told in the course of our lives “It’s all relative?”
I hear that over and over. And I suppose that there’s a lot of truth in that statement. If we push aside considerations of moral issues, the “Non-Einsteinian Theory of Relativity” probably holds true in many cases.
When I was a child, one of my father’s Uncles, Dr. Wilfred Gallay, was a renowned chemist and the head of the International Union of Pure and Applied Chemists (IUPAC). He lived in Ottawa, Canada, but in his fairly frequent visits he made a strong impression on me. In fact, his stories of DNA and cloning probably set me in motion for a career in biomedical sciences. But great Uncle Wilfred, who by the way patented a new type of glue that enabled the Allies to manufacture planes far more rapidly, left me with a legacy of his own personal “Theory of Relativity.” My own relative’s relativity theory.
My great uncle, a highly competitive man by all accounts, once made the following statement to me: “It doesn’t matter what grade you get in school or university, as long as it’s higher than that of everyone else.” Hence, his personal “Theory of Relativity.”
Why I am suddenly sounding off on the relative nature of things? What set me off? Allow me to explain.
Over the past 6 months I have received a number of international grants to review. Unlike my jobs reviewing grant proposals in the US system, which usually include a heavy stack of proposals (10 or more per review session), most of these international grants arrive as a single proposal to review. Sure, it’s easier than reviewing 10 of them, but it means that the reviewer has absolutely nothing to compare them with.
To their credit, some of these agencies supplied an online critique form that explicitly gave instructions to the reviewers to say “Should be funded” or “Should be funded if there are sufficient resources” and so on. Others, however, had an arbitrary scoring system leaving the reviewer (me!) guessing as to what that means.
So how would someone, having received a single proposal to review, have a basis for judging whether a grant should score “Top 1%, Top 5%, Top 10%” and so on? And what does that mean? If the proposal is great, and I score it top 5%, is that a mark of complete approval? Or are only first percentile-scored grants funded?
This is very frustrating for a reviewer, and I’m sure even more frustrating for the applicants. So for those of you on the other side of the pond who have any influence on your funding agencies, it might be worthwhile to use it!
As for me, while great Uncle Wilfred’s “Theory of Relativity” may seem rather cold-blooded to some, my respect and appreciation for him continues to grow.
I guess they want to know, “in your great and esteemed experience of evaluating grants, what percentage does this fall into”? Not great, but I’ve seen this sort of request before – as when people ask me for a reference for someone I’ve supervised.
Not great, as you note. But to complicate matters even more, this reminds me of the reference letter issue that I recently blogged about where I’ve pointed out differences in that US recommendation letters tend to gravitate towards the superlative with weaknesses merely omitted (whereas many European letters are more direct and to the point). With grant proposals the same is true. US grants tend to be “1/3 completed” when proposed, whereas European grants are more exploratory. This makes it even more difficult to score foreign grants, and more likely that an unaware American reviewer will not sufficiently appreciate European proposals.