NIH's and NSF’s Rigor and Transparency efforts are intended to address an increasing trend of reports of failure to replicate important basic/preclinical studies.  

National Institutes of Health

What has already been implemented?

Requirements for applicants to, and reviewers of, NIH research awards

In 2015, NIH released a new policy focused on enhancing reproducibility of research funded through NIH: Implementing Rigor and Transparency in NIH & AHRQ Research Grant Applications [NOT-OD-16-011]

The policy introduced new requirements for applicants to, and reviewer of, research grants and mentored career development awards. The requirements apply to the full spectrum of research, from basic to clinical, and focus on four areas deemed important for enhancing rigor and transparency:

1) the scientific premise forming the basis of the proposed research,
2) rigorous experimental design for robust and unbiased results,
3) consideration of relevant biological variables, and
4) authentication of key biological and/or chemical resources. 

See NIH’s one page guide one these four areas, as well as related FAQs, and blog post. See our Resources and Trainings page for additional resources

What’s next?

Formal Training in scientific rigor and transparency

In December 2015, NIH released an Advanced Notice of Coming Requirements for Formal Instruction in Rigorous Experimental Design and Transparency to Enhance Reproducibility: NIH and AHRQ Institutional Training Grants, Institutional Career Development Awards, and Individual Fellowships. [See NOT-OD-16-034]

This training requirement is to be implemented no sooner than 2017. NIH and AHRQ will issue a Notice at a future date to provide an updated timeline for implementing this requirement. When implemented, applications will be expected to provide the following:

  • Institutional training grant applications will be required to include within the training program plan a summary of the instruction planned for all predoctoral and postdoctoral trainees to ensure the knowledge and skills required to design and conduct rigorous, well-controlled experiments that consider all relevant biological variables, use authenticated biological and chemical resources, and apply appropriate statistical tests for data analyses.  In addition, a separate attachment will be required to describe in more detail the instructional content and curricular content. 
  • Institutional career development applications (K12/KL2) will be required to include within the career development program plan a summary of the instruction planned for all scholars to ensure the knowledge and skills required to design and conduct rigorous, well-controlled experiments that consider all relevant biological variables, use authenticated biological and chemical resources, and apply appropriate statistical tests for data analyses.  In addition, a separate attachment will be required to describe in more detail the instructional content and curricular content. 
  • Individual fellowship applications will be required to summarize in the research strategy section plans to ensure rigorous, well-controlled experiments that consider all relevant biological variables, use authenticated biological and chemical resources, and apply appropriate statistical tests for data analyses.  In addition more detailed description of instruction in rigorous experimental design to ensure reproducibility will be required in the section on Institutional Environment and Commitment to Training. 

National Science Foundation (NSF)

NSF is exploring policy changes in support of Improving Reproducibility, Replicability, and Robustness in Funded Research. NSF issued a guidance document in which it discusses a framework to improve reproducibility, replicability, and robustness in NSF-funded research, intending to improve quality of question formulation, data collection and analysis, and dissemination of studies yielding negative results.

NSF’s “Dear Colleague Letter” announced rigor & reproducibility as a focus for proposals in Computer and Information Science and Engineering, and the Directorate for Geosciences

However, at this time, there is no specific requirement within NSF proposals to specifically address rigor and reproducibility, unless explicitly stated in the FOA.

  • In addition, NSF has several policies covering Dissemination and Sharing of Research Results , including NSF’s data sharing policy, NSF’s data management policy, and data management requirements relevant to specific directorates and divisions.