For NIH-Funded Studies

Research applications are reviewed and scored by a group of scientific experts from the extramural community in a process called peer review based on significance, investigators, innovation, approach, and environment. RFAs and PARs often include additional nonstandard review criteria. In addition, all applications that include an NIH-defined clinical trial have specific review criteria, including study timeline; milestones; resources and data sharing plan; protection of human subjects; inclusion of women, minorities, and individuals across the lifespan; and addressing potential biohazards. For applications that include distinct phases, such as the UG3/UH3 mechanism used by the NIH Collaboratory Trials, reviewers assess the clarity and appropriateness of proposed milestones that must be met for the transition from the first phase to the second.

In response to concerns regarding the complexity of the peer review process for research project grants and reputational bias impacting peer review outcomes, the NIH is implementing a new, simplified review framework starting January 25, 2025. The new review criteria are intended to help reviewers hone in on the most important questions to determine the merit of research projects. The new criteria reviewers will consider are as follows:

1. Importance of the Research (Covers Significance and Innovation), scored 1-9
2. Rigor and Feasibility (Covers Approach), scored 1-9
3. Expertise and Resources (Covers Investigator and Environment), scored sufficient or insufficient

Reorganizing the review criteria into broader factors simplifies the process and minimizes the potential for reputational bias. Before this change, peer reviewers were responsible for assessing policy compliance of research, which could distract them from determining the actual merit of research. The new framework is expected to have minimal impact on how applications are written.

For PCORI-Funded Studies

PCORI evaluates LOIs for administrative compliance and programmatic responsiveness and subsequently notifies applicants of a decision to advance their projects to the full application stage (Figure). If the full application is reviewed and approved to proceed to the next step, it moves into the merit review process. Merit review consists of a panel of external experts including patients, clinicians, and researchers who evaluate applications to determine adherence to PCORI’s Methodology Standards, appropriate human subjects protections, and the merit of the study. The proposals also go through selection committee review (a sub-set of the board of governors) to ensure the proposals meet the mission of PCORI and that the results of the study can penetrate into practice. Following review, the panel holds discussion and scores the application. The slate of applications is reviewed by the Standing Selection Committee, then advanced to the Executive Director for approval. All applicants receive detailed feedback from merit review that supports resubmission for those who did not receive funding.

Figure. PCORI Funding Cycle

Funding Announcements are offered in 3 cycles across the year with standard due dates, although extensions for cause of up to 24 hours may be requested.

Inclusion of patients/stakeholders in the entire research process, including early and ongoing engagement, is necessary for PCORI funding. These crucial partnerships can fill knowledge gaps, improve scientific merit, and most importantly, enhance ability for results to be implemented in clinical settings.

Common Pitfalls Seen in Applications

• Overly ambitious—proposes research beyond the length of the application
• Missing or inappropriate comparison groups (for more information, see the chapter on Experimental Designs and Randomization Schemes)
• Lack of sufficient expertise or skilled collaborators needed to complete the studies
• Lack of diverse sites to ensure generalizability of findings
• Insufficient publications in the area of proposed studies
• Studies that lack appropriate statistical power to address the primary hypothesis
• Lack of documentation and experience needed to recruit the population for the study