Research and Funding
Inside NIA: A Blog for Researchers

Strengthen your research plan for a better score – Dos and Don’ts

Strengthen your research plan for a better score – Dos and Don’ts

As an NIA program officer for 11 years and counting I have the rare privilege of monitoring many study section meetings devoted almost exclusively to R03, R21, and R01 applications. (Don’t know what these numbers mean? Start here.) One thing that leaps out of this experience is that a perpetually changing cast of peer reviewers raise the same basic criticisms over and over again.

What are these pitfalls? Of course, I can’t list every possible thing that reviewers can find wrong in an application, and my experience is limited to observational studies of aging human populations. Still, the list below is general enough that some or all may apply to you even if your research if far from this field.

Most importantly, simplify your research plan.

When applicants come to me for advice after receiving a disappointing score, the most common recommendation that I make is to simplify the research plan. In fact, I have the word “SIMPLIFY” posted on my office wall in large letters. It’s good advice for all of us.

Scientific justification, writing style, page limits

DO

  1. Justify the proposed research scientifically, including theoretical motivations, relevant published data, and pilot data if appropriate.
  2. Obvious potential overlaps with existing grants should be thoroughly addressed.

DON’T

  1. Don’t skip the literature review entirely or ignore large chunks of the relevant literature in order to save space.
  2. Don’t use a writing style that is dense and confuses the reviewers. Not sure if your style will annoy reviewers? Ask someone you trust to take a look. Employ a reader, someone who is not in your scientific field but can read your draft and advise you about what is not clear. That person can really help with presentation. And, look over our science communication advice.
  3. Don’t place critical information in an appendix in order to get around the page limit.

Specific aims

DO

  1. Applications become stronger by reducing complexity and eliminating poorly developed aims from the proposed research. How many aims should an application have? With current page limits, it’s difficult to adequately defend more than three or four.
  2. Select aims that are novel or fresh, and are capable of substantially advancing the field.

DON’T

  1. Don’t include untestable aims. Aims must be testable to make it into your application. If you lack equipment or knowledge to test an aim in this application, then plan it for a future one. Don’t include it in this one.
  2. Don’t include aims devoted to building research infrastructure if the infrastructure is not essential for the work you propose. Don’t assume that reviewers highly value an application just because it builds research infrastructure. We have research infrastructure opportunities. If you want to write one of these then talk to your program officer. Do not include that in an R01.
  3. Don’t include aims that lack specificity or suggest a fishing expedition. Write a “tight” set of aims. Like a good story, the parts connect, and the conclusion follows from them. (Of course, you don’t know the ending yet!)

Key personnel

DO

  1. Key personnel must have appropriate expertise and experience, specific to the stated aims.
  2. Keep in mind that the expertise and experience of the key personnel, even if outstanding, cannot offset weak aims.

DON’T

  1. Don’t assume that it’s irrelevant whether key personnel have a history of successful collaboration together. It really does help if the key personnel can show that they have worked together successfully.

Data collection

DO

  1. When it comes to data collection activities and the analytic plan, they need to be linked to the stated aims.
  2. Using multiple sites for data collection? The scientific need for using multiple sites must be clearly justified.
  3. Use preliminary data to show feasibility of aspects of the research design—it’s important!
  4. Be clear about the exact number of participants expected to be enrolled in the research project.

DON’T

  1. For competing renewals, don’t add new participants if the stated aims can be adequately addressed without them.

Have I covered the most common pitfalls? Are there others you would include? Or, do you have questions about those I’ve shared? Please get in touch with me by commenting below.

 

Read next:

How to avoid annoying your reviewers—tips from review, part 1

13 Comments
Share this:
Email Twitter Linkedin Facebook

Posted by Anne M on Jan 28, 2015 - 2:27 pm

Can you provide or point to a source for specific guidance in submitting a competitive renewal for an R01? I've been unable to find any information on this. thanks

Posted by Dallas Anderson on Jan 28, 2015 - 5:18 pm

If you are thinking about a competitive renewal, it is important that you do not let your current grant expire before you submit the application. In terms of the science, please discuss your proposed aims with your program officer and get feedback about their value and whether a competitive renewal makes sense, or whether a new application is more appropriate. Finally, note that the submission deadline for a competitive renewal R01 application is different from a new application (see http://grants.nih.gov/grants/funding/submissionschedule.htm).

Posted by Nancy on Jan 28, 2015 - 3:13 pm

This is a very useful list. Don't know if it is relevant for NIA, but at The Retirement Research Foundation, we also encourage applicants to include a section on likely next steps. It should reflect how those next steps might vary depending on the pattern of findings from the study. We find that it is a good method of identifying investigators with the strongest commitment to their fields of inquiry and gives us a better vision for how our investment might play out down the road.

Posted by Dallas Anderson on Jan 28, 2015 - 5:09 pm

We don't have a next steps section and sometimes the next steps depend on the research outcomes but explaining follow-up steps is a good strategy in any case.

Posted by Mary on Jan 29, 2015 - 4:39 pm

If I may - nobody can object to your injunction to applicants to SIMPLIFY; but the reality is that life is complicated, and one cannot address the complexities by oversimplifying them. The art then is to present complex things in a simple way, and we try to do exactly that, in grants, manuscripts, oral presentations. But, to achieve that, one has to stick to the main principles and not get bogged down in details; and/or, have enough space to write pleasant, easy-to-read prose. When we are up simultaneously against space constraints and reviewers' demands for MORE DETAIL, this becomes a death spiral.

Posted by Dallas Anderson on Jan 30, 2015 - 9:36 am

Thanks for this thoughtful comment, Mary. It is indeed a challenge to make the case for your proposed aims in sufficient detail in a limited amount of space.

Posted by Lewis H. Kuller, MD, DrPH on Feb 02, 2015 - 9:57 am

I believe that there is a far greater problem. First, the study sections fail to include enough quality epidemiologists, especially expertise in epidemiology methods and not statistical jargon. Second, most important, there needs to be a separate NIH epidemiology-etiology study section that focuses on dementia and AD. The recent advances in measuring the brain, PET-amyloid, MRI, cerebral blood flow, and vascular disease are not refereed by experts in current study sections related to epidemiology and population sciences. The study of only ‘dementia’ in epidemiology studies is pretty much dead, like studying heart disease without clinical evaluation, i.e. rheumatic heart disease, hypertension, Chagas, atherosclerotic changes, etc. There is an unusual opportunity for epidemiology studies, including genetics, to enhance our understanding of etiology and prevention of AD, neurodegeneration, vascular disease and markers of aging. If the current amyloid clinical trials do not show big effects on reducing AD, especially in the elderly, then we are likely back at square one. I think there needs to be frank discussion about the future of epidemiology research in dementia and aging, given new technology, and opportunities. I doubt the excellent suggestions you have made in your blog will change the ‘climate’ of the study section or move the field forward.

Posted by Dallas Anderson on Feb 02, 2015 - 4:17 pm

Thanks, Lew. I agree that the field of dementia epidemiology is quickly evolving as you suggest, and this has implications for recognizing impactful research, organizing study sections, and finding appropriate peer reviewers. The dos and don’ts that I have provided are not just for beginners; they are also for senior investigators who wish to propose exciting research based on rich databases.

Posted by Bud on Feb 02, 2015 - 2:27 pm

Having served as a reviewer on a number of CSR and Program study sections over the years I think your suggestions should not only be helpful for the applicants but also will be a boon to reviewers. Reviewers are really in a critical position, being responsible to evaluate applications scientific merit and impact on the field; if they receive an application that convoluted and confusing rather than clear and concise, then chances are that they may miss a key point and score an application lower than the true research issues involved in it should merit. The emphasis now on trying to develop topics that have "impact" and will "make a difference" is really critical to thinking through the writing of an application. "Novel" statistical or lab techniques are essentially a waste of effort unless their output really adds something. We really cannot be comfortable with writing applications that survey the same established risk factors or socio-demographic indicators, in essentially the same way, but in perhaps a slightly different setting. While this helps a little when there are equivocal findings to settle, epidemiology really should be aiming higher to identify etiologic factors. Too often we settle for obtaining and replicating the stock "epidemiology of disease" statement : "It is the Nth leading cause of death, affects X millions in the US, more men than women, and more of this ethnic group than that, while also increasing exponentially with age..." In epidemiologic study of diseases where most the above is known, we really must write applications that will have an impact on the disease in a much more specific way. Your suggestions to Grant writers will give them the framework to do that. Now they must have the ideas and present them to study sections in a fashion that can be recognized. Further study section reviewers need be focused on identifying similar things.

Posted by Dallas Anderson on Feb 02, 2015 - 4:18 pm

With the tight paylines, reviewers are not impressed with proposed research that would, at best, lead to an incremental advance in the field, and they are impatient with confusing or unfocused applications that are tedious to read. Thanks for your illustration of these fundamental points.

Posted by Fred on Feb 02, 2015 - 6:17 pm

This is a good succinct list of do’s and don’ts. I sent the link to a junior faculty member in our department, but it is a good reminder for seasoned investigators as well!

Posted by Walter A. Rocca on Feb 05, 2015 - 11:45 am

Reading the list of suggestions for grant writers that you prepared prompted me to think about some broader issues in peer review of NIH grant applications. My impression as an unhappy “perpetrator” of reviews and as an unhappy “victim” of reviews over many years is that we are living through a crisis of peer review. The job of the peer reviewer has become impossible and the job of the applicant has also become impossible. Where does the crisis come from? I believe there are two important negative trends. One trend is economical. When the NIH funding line is at 20 or 25%, the peer reviewer knows that the important and non-controversial applications will be funded, and that there will be room for some more innovative and less conventional projects to open new possibilities and new directions. When the pay-line is below 10% or below 5%, the peer review becomes mainly a political problem. Only the most prestigious institutions will receive funding and only main-stream aims will be considered fundable. Any new idea or new direction will be considered risky and will not be funded. The scientific community is in a financial and political freeze. In absence of adequate funding the peer review system does not work. The second trend is grounded in philosophy of science and in economic, political, and cultural influences on science. As discussed by Lew and Bud above, neurosciences are in the midst of two major technological transformations, one related to genetic technologies and one related to neuroimaging technologies. These changes have created a major expectation that progress will come from the application of these technologies. The new NIH position is to fund “mechanistic grants” or “grants that advance the understanding of the genetic, biochemical, or anatomical (structural) basis of neurological diseases”. This position is called in philosophy of science “biological reductionism”. The simple idea is that the diseases of the American citizens are due to alterations at the subcellular, cellular, or tissue level of the body (or brain), and that the health of Americans will be improved by investing our tax payer money in mechanistic research and in biotechnology. This trend is partly related to the first trend because in absence of public funding (NIH and tax payers) the funding has come in part from the partnership between the public (NIH) and the private sector (pharmaceutical industry, biotechnology industry, and information technology industry). I believe that the public agenda and the private agenda do not overlap. Population studies, observational studies, epidemiology, clinical research, and public health in general will be degraded unless there is a profound reorientation of research,. I do not believe that technology per se will solve our public health problems. My comments may not help grant writers to get funded, but may suggest a different level of discussion, if anybody cares.

Posted by Jim Maraganore on Feb 06, 2015 - 7:22 am

This is useful advice, Dallas. Your "simplify" mantra always made our co-authored papers better. The current 12 page limit (versus the old 25 page limit) is a constraint. The current structural requirements for the research strategy ("significance, innovation, approach") are opaque (versus the old requirements: "aims, background, preliminary studies, methods"). The current 9-point scoring system is complex (versus the old 5-point scoring system). There is a plethora of RFAs with many complex requirements. What are some of the do's and don'ts to successful grant writing in the recent 5 years, versus in the past decades, for those of us who took a break but are now looking to get back into the funding game? I echo some of the comments made by Walter Rocca above. We are in an era of expensive technologies, biomarkers and endophenotypes, big data, big NIH, small pay lines, etc. Whither the simple, pragmatic, and clinically meaningful study? Simple writing yes, but simple science no, with complex grant requirements and scoring schemes and funding mechanisms, seems to be the Gordian Knot. Can this knot be untied, or like Alexander how can we simply cut through it?