The missing links: Why NIH doesn’t allow hyperlinks in grant applications
Hyperlinks and URLs (or Uniform Resource Locators – who knew?) are a ubiquitous part of working or playing online. From shopping to social media and even NIH Guide Notices, our modern world is one big hyperlinked, interconnected information ecosystem.
So, what’s up with NIH being so strict about hyperlinks in grant applications? It all boils down to some of the core values of NIH’s Peer Review system (PDF, 773K).
Fairness, confidentiality, and integrity
We strive to ensure that review criteria are applied fairly to each grant application so that no applicant receives an undue advantage. This means we pay attention to seemingly mundane but important distinctions such as page limits, permitted font sizes, or attempts to circumvent section space limitations. Allowing extra material in the application, accessible via hyperlinks, would unfairly benefit some applicants over others. An applicant would (rightfully!) be upset to learn that competing applications under review provided unauthorized additional material that reviewers accessed through hyperlinks.
The integrity of NIH’s review system must always be protected to maintain public trust. However, a simple click on a link in a grant application can be tracked to the IP (Internet Protocol) address of the reviewer, unmasking their confidential identity. Even if this does not reveal the reviewer’s exact identity, one could make an educated guess as to whom the application is assigned based on the institution from which the IP address originated. This could compromise the integrity of the NIH peer review system and NIH takes all breaches of review integrity seriously.
Do’s, don’ts, and a few exceptions
The NIH Office of Extramural Research assembled a list of the Do’s and Don’ts of Hyperlinks, stressing the aforementioned points. But it is also important to understand there are a few exceptions to the “no hyperlinks” rule:
- If the FOA explicitly permits hyperlinks, then you may include them. (Be sure to always read the entire text of the FOA.)
- NIH permits hyperlinks to your “MyNCBI” profile in your NIH biosketch and in the publication lists as commented on in the Hyperlinks and URLs section of the Format Attachments page.
- Further, if hyperlinks are permitted, in the interest of transparency, the full URL must be presented and appear in blue, as in https://www.nih.gov.
- Of note, there are no special permissions for links to NIH or government websites. In other words, you don’t get a free pass by linking to us. There is also no explicit allowance for links in application sections, such as “Facilities and Other Resources” that do not have page limits.
So I hope this helps explain why, for the above reasons and those specified by NIH, that unless the FOA explicitly states otherwise, the inclusion of even a single inappropriate hyperlink can represent a major violation of the rules governing NIH peer review and can lead to an application being withdrawn from consideration.
So, before you copy that link to your institution’s facilities, lab website, or that new cool data in a big, online database into your next application, please ask yourself whether it’s important enough to risk your application being withdrawn from review. Double check the FOAs and NIH guidance to make sure if the hyperlink you have in mind is allowed. If not, please respect the boundaries set by NIH grant rules and general principles, which help us ensure that all applications receive equitable, fair review. If you have questions, contact the NIA SRB or leave a comment below.
Comments
Thank you for your clarification on the risks of using hyperlinks in grant applications. Your blog was very informative and something I would have not thought about could potentially come in the way of our application review.
Thank you Preeti for the kind words and for subscribing to our blog! I’m glad this has been of use.
Thanks very much to Greg for explaining this policy. It is absolutely depressing and tells us that grant review has not even entered the 20th century. We simply cannot prosper if we continue to rely on pre-internet ways of doing business. It would be quite straighforward to set up a secure system by which links could be be fully anononymized (think bitly) for inclusion in grants with no danger to reviewers. The problem is a lack of imagination on our part - the reviewers and scientific community- with no criticism intended for Greg and his hardworking colleagues.
Thank you Peter for your comments and recommendation. The NIH is always looking to improve and update our policies and practices, and to do so efficiently, we need to hear directly from the extramural community.
I agree that this is a good idea. However, these are some barriers to equitable implementation.
There has been a lot of confusion on this. Some grants offices have provided information that as long as the link is not "live" (i.e. clickable), it's OK to cite www . something . something, because if the reader wants to follow up, they will have to actually look it up. Same as with any other reference.
The problem is that with new versions of commonly used software (including MS Word and Acrobat) the software will recognize and search for the link, and make it live. This is because the defaults are set that way, and it happens automatically in the background. And the software does this without making the link blue, so there is no way to know unless you hover over it.
Thanks John, this is an important challenge you’ve noted and it’s good that you’ve raised some public awareness so it is on the radar of researchers everywhere.
From the guidance from NIH and this article, it sounds like I can paste in a URL as text (https://...), but I just can't add a hyperlink to it. Is that correct? NIH seems to be equating the two, but they are actually different entities. It would be great to clarify this point. Even the start of this article doesn't make it clear if URLs as text are not allowed. Obviously a reviewer pasting the URL into their browser will go to a webpage, the same as if they clicked on an embedded hyperlink. Thank you for a response. :)
Thank you for your clarification question! When permitted, the actual URL must be hyperlinked (as per instructions in the SF-424 seen here, bullet #4, with examples).
Thanks for that reply, Greg! Very helpful clarification as I know a number of investigators who are under the impression that the hyperlink is the issue, not the URL itself. Thus, they have been inclined to keep them. This is a particular issue for computational folks who are developing web resources/databases and want to be able to include the URL for the web site portal for the resources, as well as cite resources they have previously that are hosted on web sites. So would they just refer to the web site name without the "https//:" (and without "www" or ".org")? Thanks for helping us all to understand this policy better. :)
Thanks for the question Carmen, I’ll do my best here. I need to be careful not to exceed the boundaries of my role as an SRO. (Afterall, I’m a review policy educator or ‘enforcer’, not a review policy ‘maker’!), but if the information that the hyperlink would reference or lead to is relevant to the review of the application, that material should probably be in the application rather than accessed via hyperlink (for all the reasons stated above and in the official policy). I’d also orient you towards the first two bullet points in the official policy, highlighting this concept. The major takeaway should really be to avoid the use of hyperlinks unless specifically permitted for all the reasons enumerated in the blog and in the ‘Purpose’ section of the official policy. I hope that helps!
In the NIH Biosketches it is encouraged to provide PMIDs with each reference listed. People (up til now) usually linked these to the journal web site. Is NIH telling us to remove these links?
Example ( the PMID linked to the journal web site):
c. McDonough AA and Youn JH. Potassium homeostasis: the knowns, the unknowns, and the health benefits. Physiology, 2017, 2Mar;32(2):100-111. Comprehensive Review. PMID: 28202621
Hi Alicia and thank you for your question!
The official wording on this topic from the SF424 is that “Hyperlinks and URLs are only allowed when specifically noted in funding opportunity announcement (FOA) and form field instructions.” So if both the FOA and the form field instructions for the submission of that application in response to the FOA ask for the PMID for references to be hyperlinked, then it would be permissible. Otherwise, this information should not be hyperlinked.
We just had an application administratively withdrawn because a hyperlink was included in the biosketch that had "tracking/privacy concerns where there is an indication that anyone named in the application would have access to the tracking data for the link, XXX, as it is the personal website of a Key personnel investigator." I can't find guidance anywhere in the application guide or FOA that says we cannot include personal websites. Can you provide some clarity?
Hi Alicia, and thank you for your question.
Each FOA will only identify if and when a hyperlink is permitted for use (and will enumerate what may be hyperlinked, if permitted).
But the permission for use of hyperlinks in FOAs is rare, and the NIH application guide specifies the terms in the Format/Attachments section. Any hyperlinks that have the potential to reveal who has clicked them has the chance to reveal reviewer anonymity, and should not be used, as noted in this posting as well.
How does one reference a website in a Bibliography section, without using a url?
Thank you Paul for this question. The SF424 instructions on using Hyperlinks and URLs states that “The use of hyperlinks is typically limited to citing relevant publications in biosketches and publication lists. “, and the ‘Do’s and Don’ts of hyperlinks’ uses the same language when describing “Do use hyperlinks in relevant citations and publications included in biosketches and publication list attachments.”
The ‘publication list’, in this case, would include the references/citations/bibliography. As always, please also be mindful of the other rules, guidance and intentions behind when to use a URL/hyperlink, and when not to.
Greg,
Thank you this blog. I have three questions:
1) How should web resources be listed in the "Literature cited" section of RO1 proposal? Can you please give specific examples of what the text looks like?
We would like to reference sites that do not provide publication-type citations, such a github.com, synapse.org, and openneuro.org?
2) The submission process added links in our proposal where there were none in the original document. Links appear out of the blue in the eGrant. Some of these added links don't go to websites as they were falsely identified as URLs by the NIH pdf assembly process.
What is the trigger that causes the pdf assembly to add links where there were none in the original document?
3) Does this automated hyperlink process happen to citations in the "Literature cited" section also? How can we avoid having automated hyperlinks added while still referencing critical online databases and software resources in literature cited?
Elaine Bearer, MD-PhD
Professor
University of New Mexico and Caltech.
Hi Elaine, thank you for your questions. Regarding citing web materials in your references, I’d recommend consulting this grant resource on citations, specifically the ‘Citing Medicine’ link from the National Library of Medicine.
While I cannot speak to grant assembly or automated hyperlink generation from use of software, I can point you towards the language in guide notice NOT-OD-20-174 and other NIH resources on the use of hyperlinks that identify where the use of hyperlinks are typically limited to, and note that page-limited sections such as Specific Aims, Research Strategy or other sections would not generally be permitted.
Add new comment