I received the following email last week:
“Hello All:
As you are aware there has been some discussion about the relevance of the form 100 as part of the grant application package. At the Department Head meeting yesterday it was decided to retire Form100. So that means starting today, the Peer Review Form 100 is no longer required as part of your grant application signature package”.
and there was much rejoicing in the land of grant wranglers!
Seriously – within five minutes I’d sent two and received three celebratory emails to/from fellow sufferers. At a seminar later that day, with lots of grant wranglers in the audience, there was a round of applause and an outbreak of grinning and thumbs-upping at friends when the news was repeated. One colleague has suggested that we throw a party to celebrate Form 100’s demise, and I don’t think she’s joking.
Ding dong, the pink form's dead!
Until today, obtaining institutional sign-off of any grant required the submission of the abstract, budget summary, budget justification, an internal cover sheet signed by the PI and their department head (printed on blue paper)…
…and two copies of the internal peer review form (printed on pink paper), each one signed by a different PI who was not a co-applicant or collaborator on the grant.
In theory, the latter requirement ensured quality control and pre-submission exclusion of no-hoper grants. However, in practice, grants were not reviewed thoroughly enough (or early enough) to make a real difference. Whenever a big deadline approached (and for some of the big federal competitions the PIs in our building will submit literally hundreds of applications between them), the requirement for two peer reviews of every grant created a frustrating bottleneck: our internal deadlines typically fall a week or so before the funding agency’s deadline, so everyone would still be working on their proposals and no-one would have a final version ready for review; and almost every potential reviewer would be frantically trying to finalise their own grants, leaving them little time to spend reading their colleagues’ drafts.
The process was stressful on the grant wranglers, too. We’d all spend far too much valuable editing/proofing time running around making deals with PIs and with our counterparts in other departments (“my PI will review your grants if your PI reviews mine”), not to mention trying to find out if there was any pink paper left anywhere in the building and, if not, whether pale purple would be acceptable.
I think internal peer review can be a very good thing, if it’s done right – and in my opinion, right means selectively. Established PIs with a good track record of funding should be left alone to judge their own grants’ merits, whereas newer PIs, or those without much recent grant success, deserve a much more thorough and thoughtful review of their grants at a much earlier stage of the process.
I’m sure we haven’t seen the end of pre-submission internal grant review at our institution, although I have no idea what form it might take in the future. I’m expecting that the PIs and grant wranglers will be asked for our input, and so, dear blogosphere, I’d like to hear about the practices in place at other institutions around the world:
- Does your institution have any kind of formal pre-submission peer review of grant applications?
- If so, at what stage of the submission process is it applied?
- Do all grants go through this process, or only those submitted by new investigators?
- If the process is selective, what are the criteria for a grant requiring review?
- Do you find the practices in place at your institution to be helpful?
Thank you in advance!
1. No in practice, informal in theory
2. When the damn thing is written (this is not a good thing)
3. All in theory, whomever has the time/inclination but no deadlines in the UK system in the same way as CIHR/NSERC so no bottleneck.
4. PI who wants a review, solicits review, director of research institute looks through before submission with right to refuse to submit
5.Not particularly. As a new PI, the times my colleagues have read my grants and offered suggestions have been invaluable and I thank them for their time. However, like most new PIs I didn’t ask for that feedback early enough in the process (i.e. the ‘draft grant’ was too close to finished to really benefit from constructive suggestions, but I did my best in the time available). My advice to new PI or grant wranglers is to get feedback on the grant as early as possible so that there’s time to digest any suggestions before reaching a submit/don’t submit deadline. Easier to get a draft looked at 3 weeks before with time to fix, than 1 week before and have 6-12 months to fix.
I would say that a system like your form 100 would be appropriate if and only if all grants went through it – it isn’t mentoring for new PIs, but quality assurance. Grant mentoring like what I rant about above is different. All PIs should go through it, but only if it is a meaningful process – does the reviewer know the details of the call for submissions etc. On the other hand, with grant wranglers like yourself, you check through a lot/all of the details, so its not just the PI who has worked on it – reduces the chances of ‘silly but will get you rejected’ errors. As long as someone checks it through before submission, I don’t really care about forms!
Yeah, I’ve pretty much never seen anyone ask for feedback anywhere near early enough…
I should probably have mentioned that every grant I’ve ever worked on has had at least two co-PIs, often more than that – I’d say that four or five would be average, whereas some grants have up to eight or nine co-applicants. So there are always multiple people checking each grant for obvious errors – all the PIs plus various departments’ grant wranglers! However, the modular approach makes meeting internal deadlines even harder, as some PIs will see the value of it more than others, and all the different sections get completed at different rates!
We do for biomedical grants.
It has deadlines – you must request review 6 weeks before the submission date and get your draft in a month before it, and the reviewer(s) will get it back to you within two weeks.
It’s opt-in. Any PI can use it if they choose, but nobody has to.
I find it very helpful. I’m always grateful for pre-submission advice, it’s good to have this deadline, and I don’t have to try to line up the reviewers myself (and they get a $100 bookstore gift card as a thank-you).
Optional is good! But I can’t imagine getting one of our grants ready so early for an internal, optional deadline, as there are too many “I can only write when I have an urgent final deadline” writers around!
Torn. A deadline is a deadline so if yours is moved internally, who cares, right? But at this point I would resent the obligation for my own writing.
Jr folks could definitely benefit. Voluntary systems are flawed b/c n00bs fear looking ridic in their local environment more than at study sections (at times, ymmv). As far as peer burden goes, it would depend on the load but right now, I just want our dept junior folk funded. There are enough spillover benefits that it is well worth my time to help out. The margins for error in NIH land are much narrower than when I got my start.
The junior PIs I know seem, outwardly at least, to be more concerned about getting funded than looking silly in front of their more senior colleagues, but that would definitely depend on the local environment!
I believe the CIHR paylines are still slightly more generous than at the NIH, but it’s very tight here, too, and getting tighter.
YAY! For the retiring of your form.
As for today’s quiz:
1. Does your institution have any kind of formal pre-submission peer review of grant applications?
Yes indeed. Now ask me if it’s useful. Go on, I dares ya.
2. If so, at what stage of the submission process is it applied?
Ideally, you’re supposed to submit the form (similar to your now-dead “Form 100”) SIX WEEKS before the grant deadline. Just imagine how often this actually happens.
3. Do all grants go through this process, or only those submitted by new investigators?
Allegedly, all of them, except if you claim they are exempt. “Exempt” sometimes can have the alternate meaning “I am a busy and important researcher who has better things to do than submit this form, just see if you can make me, nyaa nyaa nyaa”.
4. If the process is selective, what are the criteria for a grant requiring review?
No real clue, but see answer to question 3, above.
5. Do you find the practices in place at your institution to be helpful?
*COUGH SPLUTTER* sorry, something stuck in my craw there.
What excellent answers! I sense a comment of the week coming your way 🙂
I think there’d be many takers for that exemption clause among the PIs I work with. The six week deadline? Not so much.
Does your institution have any kind of formal pre-submission peer review of grant applications?
Some departments (not mine) have this, but it is largely viewed as another bureaucratic hoop to jump through rather than anything useful.
If so, at what stage of the submission process is it applied?
Anytime before submission 2 signatures are needed.
Do all grants go through this process, or only those submitted by new investigators?
All. On the other hand, there is an outstanding program that I am a part of to help mentor new investigators, and here they need to plan well in advance to have their mentors read their proposals.
If the process is selective, what are the criteria for a grant requiring review?
N/A
Do you find the practices in place at your institution to be helpful?
The mentored investigator program is great. I had an award myself as a new investigator, and now serve to mentor other new investigators. This system has a high level of success.
The other “signing off on grants” is just another bureaucratic pain in the pipette.
“but it is largely viewed as another bureaucratic hoop to jump through rather than anything useful.
This.
The mentorship program, on the other hand, sounds great. I think we should set up something similar for our new PIs, and I may well drop you an email asking for more details if/when we’re all asked for our input!
As I think about this more (tangentially related re young staff getting grants), the best help for writing grants when you haven’t done it before isn’t in peer review, its conversations when you’re trying to decide the shape and context of the work. How much can you expect to get done for $x and y years? How ambitious is too ambitious, how safe is too safe? Answers to those questions are needed long before the serious writing begins. By the time there’s a peer-reviewable draft being kicked around, budgets and ideas and plans are often too concrete to change much. This probably comes under the heading of mentoring of new staff rather than reviewing of grants but I think it would be very useful.