Research and Funding
Inside NIA: A Blog for Researchers

Payline is 13 for NIA-reviewed grant applications?

Payline is 13 for NIA-reviewed grant applications?

Buried within our NIA 2013 funding policy is the apparently shocking statement that our payline for NIA-reviewed research grant applications is 13. When the top score is 10 and the lowest possible score is 90, then a 13 is little short of perfect. We worry about this too. We know that we are leaving some truly outstanding work unpaid. How did this happen?

Which applications do we mean?

First, some important background: NIA-reviewed research grant applications include P01, U01, U19, and U34 grant applications. (Don’t understand what these alphanumeric codes mean? Better get yourself to the Types of Grant Programs tutorial without delay.) Applications for these kinds of grants are always reviewed by NIA study sections, rather than the more well-known review groups at the NIH’s Center for Scientific Review. Learn more about chartered review committees and special emphasis panels at the NIA.

What’s the difference in payline between NIA-reviewed and CSR-reviewed applications?

The payline for NIA-reviewed research grant applications is 13, out of a possible range of 10 (best) to 90 (worst). This means that NIA-reviewed applications with a score of 13 and below may likely be funded. Many CSR-reviewed R01 and R21 applications funded by us have scores worse than 13 (that is, higher than 13). The priority scores climb as high as 30! You might ask why CSR-reviewed research is apparently so privileged.

We focus on estimated success rate.

The key metric here is estimated success rate, not payline. And estimated success rate tells a very different story. Our estimated success rate for all NIA-reviewed applications this year is 18 to 23%. For program project applications it is in the range of 23 to 28%. You might be surprised, but this is really the case, even with a payline of 13! This means that a large percentage of applications reviewed at NIA receive nearly perfect scores, and we plan to fund a fifth to a quarter of applications. By comparison the success rate for CSR-reviewed research will be in the range of 14 to 17%. We can almost wonder why NIA-reviewed research is so privileged.

Sally Rockey did a great post earlier this year about the relationship of paylines and success rates. If you haven’t seen it already, please do check it out.

Now we can identify how this happened.

Our NIA-chosen reviewers are giving better scores on average to their assigned applications than CSR-chosen reviewers are giving to the NIA applications that they are assigned.

And why is that? Perhaps NIA-chosen reviewers have a finer appreciation for aging research than do CSR-chosen reviewers. Or maybe, research reviewed at NIA is of higher quality than aging research reviewed at CSR. Or it could be that reviewers chosen by NIA are more generous people than are CSR-chosen reviewers. Maybe all are true to some degree, or maybe there’s another factor we haven’t identified. If you have thoughts about what it might be, please let me know by commenting below.

How are we responding?

No matter the reason for the difference in scores between NIA-reviewed and CSR-reviewed applications, we need to pay attention both to overall success rates and to relative paylines when balancing the portfolio of NIA-reviewed and CSR-reviewed research. Because we are looking at success rates as well, at times, the NIA-reviewed payline can appear miserly.

Do you have follow-up questions? Please submit them as comments, below. I will read all comments, and may do a post in response to your thoughts.

 

Read next:

Where will my grant application be reviewed? Center for Scientific Review? NIA Scientific Review Branch?

5 Comments
Share this:
Email Twitter Linkedin Facebook

Posted by Alex Smtih on May 22, 2013 - 2:14 pm
Thanks for this explanation! For the junior researchers out there - do NIA K awards and Beeson grants count as NIA reviewed or CSR reviewed? Or are those scores counted separately?

Posted by NIA Blog Team on May 22, 2013 - 4:13 pm
Alex, NIA K applications are reviewed here at NIA, including applications for Beeson grants (K08, K23). This post covers research grants, but we'll tackle K awards and other career development awards in an upcoming post.

Posted by Anonymous on Jun 04, 2013 - 1:46 am
More commentary on career awards would be appreciated. Thanks!

Posted by Robin Jump on May 22, 2013 - 4:34 pm
Dr. Barr--thanks for this blog and signing me up. It's great to receive a friendly message through the NIA and, this week at least, a nice distraction from my medicine --> hospice service.

Posted by Georgiana Bostean on Jun 04, 2013 - 11:43 am
For new investigators like myself, it is quite helpful to find this kind of information in an accessible location and format. I actually recently have been asking myself the question of who reviews what. Thank you for starting this blog.