Skip to content

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
  • Home
  • About
    • NIH Collaboratory
      • Coordinating Center
      • NIH Collaboratory Trials
      • Core Working Groups
      • Steering Committee
      • Distributed Research Network
      • Our Impact
    • Living Textbook
      • Table of Contents
      • How to Use This Site
  • Resources
    • Data and Resource Sharing
    • Training Resources
    • Tools for Researchers
    • Publications
    • Knowledge Repository
  • Webinar
  • Podcast
  • News
    • News Feed
    • Calendar
    • Subscribe
return to home
Subscribe to Newsletter go to twitter feed go to linkedin go to blue sky feed
Search
NIH Collaboratory
Living Textbook of
Pragmatic Clinical Trials

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
home button

Rethinking Clinical Trials

A Living Textbook of Pragmatic Clinical Trials

  • Design
    • What is a Pragmatic Clinical Trial?
    • Decentralized Pragmatic Clinical Trials
    • Developing a Compelling Grant Application
    • Experimental Designs and Randomization Schemes
    • Endpoints and Outcomes
    • Analysis Plan
    • Using Electronic Health Record Data
    • Building Partnerships and Teams to Ensure a Successful Trial
    • Intervention Delivery and Complexity
    • Patient Engagement
  • Data, Tools & Conduct
    • Assessing Feasibility
    • Acquiring Real-World Data
    • Assessing Fitness-for-Use of Real-World Data
    • Study Startup
    • Participant Recruitment
    • Monitoring Intervention Fidelity and Adaptations
    • Patient-Reported Outcomes
    • Clinical Decision Support
    • Mobile Health
    • Electronic Health Records–Based Phenotyping
    • Navigating the Unknown
  • Dissemination & Implementation
    • Data Sharing and Embedded Research
    • Dissemination Approaches for Different Audiences
    • Implementation
    • End-of-Trial Decision-Making
  • Ethics & Regulatory
    • Privacy Considerations
    • Identifying Those Engaged in Research
    • Collateral Findings
    • Consent, Disclosure, and Non-Disclosure
    • Data and Safety Monitoring
    • Ethical Considerations of Data Sharing in Pragmatic Clinical Trials
    • Ethics for AI and ML
    • IRB Responsibilities and Procedures

Introduction

CHAPTER SECTIONS

End-of-Trial Decision-Making


Section 1

Introduction

Expand Contributors

Lorella Palazzo, PhD
Rachel Hays, MPH
Gregory Simon, MD, MPH

Contributing Editor

Elizabeth McCamic

Embedded pragmatic clinical trials (ePCTs) rely on partnerships between investigators and healthcare systems to address healthcare delivery concerns that are of mutual scientific and practical interest. The embedded nature of these trials makes them uniquely valuable, but also can present new or unexpected challenges compared with traditional trial design. Important practical, ethical, and scientific questions may arise for both healthcare systems and researchers at any point in an ePCT and can be especially challenging to address when they emerge as intervention activities conclude (Cho et al. 2018; Morain et al. 2023).

As the end of an ePCT approaches, research teams and their partners often face uncertainties around this undefined phase. In preparation, issues of ethics, responsibilities, commitments, and present and future relationships must be carefully considered.

One of at least 3 distinct scenarios is likely to occur (Table 1).

Table 1: Scenarios and Implications After the End of the Intervention Phase in ePCTs

Scenario Implications
Intervention is found to be ineffective. A key question is what obligations exist, if any, toward trial participants and any health system staff who were involved in trial activities? If deimplementation of trial-driven systems is needed, to what extent should investigators aid in these efforts, if at all?
Intervention is found to be effective. Critical considerations include whether any commitments between researchers and clinical partners or trial participants exist and should be upheld, and whether trial investigators ought to be involved in intervention sustainment in some capacity.
The intervention period is over but trial results are not yet known. Roles and expectations are often unclear. Trial investigators may wonder how to relate to health system partners while data analyses are ongoing. What should investigators’ involvement be as healthcare leaders decide whether to sustain or abandon the intervention while trial evidence is pending?

The period of ambiguity that occurs in the third scenario is not uncommon in ePCTs as trials often consider outcomes that take time to accumulate. Even after the follow-up period concludes, electronic data from disparate sources may take time to arrive for analysis—more than a year in some cases. Figure 1 illustrates how pathways for data collection, analysis, and dissemination in ePCTs are inherently prone to delays that can often be quite lengthy.

Figure 1. Data and Dissemination Delays in Learning Health System Studies

Figure 1. Data and Dissemination Delays in Learning Health System Studies

 

 

 

 

 

 

 

 

 

 

 

 

 

From Simon GE, et al. Healthc (Amst). 2020;8(4):100474.

Figure 1 refers to outcome data (typically quantitative) that provide evidence on the intervention’s effectiveness. Implementation data also can be collected throughout ePCTs to track fidelity, modifications, or other aspects of study execution (Proctor et al. 2011; Miller et al. 2021). However, when and how evidence becomes available and can be used without compromising the trial differs in each case. Unlike effectiveness data, implementation data accumulate as the trial unfolds and are often relied upon to address emerging challenges and improve intervention delivery.

When dealing with outcome data, the waiting period resulting from usual delays creates a relational and decision-making gray area. Investigators must successfully navigate this period to preserve study integrity (i.e., by maintaining a position of equipoise until results are available) as well as relationships with participating healthcare systems that were built and nurtured during the trial. Moreover, new issues arise when considering whether and how researchers should provide ongoing support for the intervention (e.g., in the form of training or study staff) while decisions are made.

The purpose of this chapter is to assist investigators by offering considerations for effective decision-making that honors researchers’ responsibilities and fosters ongoing collaboration with healthcare system partners while awaiting trial results. This topic will be referred to as end-of-trial decision-making.

Posttrial responsibilities comprise a overlapping and broader concept defined as what is owed to research participants (and their broader communities) at the conclusion of clinical trial (Morain et al. 2024).

Along with posttrial responsibilities end-of-trial decision-making intersects with at least two possible outcomes after ePCTs: intervention sustainment and intervention deimplementation.

Intervention sustainment refers to the trial program continuing to exist and be used in healthcare systems after trial supports and funding are no longer available (Scheirer et al. 2011; Palinkas et al. 2020).

Deimplementation is defined as the “reduction or elimination of inappropriate, ineffective, or potentially harmful health care services and public health programs” and applies to interventions that have become institutionalized as part of routine practice or organizational structures (Walsh-Bailey et al. 2021).

What must be determined (Figure 2) is whether effective ePCT interventions are maintained (sustainment) or ineffective ones dismantled (deimplementation). Not only is this determination crucial after trial results are available, but decisions made while waiting for ePCT results may be equally consequential. Continuing or stopping the intervention during the waiting period can take effort and resources to later reverse, potentially influencing the ultimate sustainment or deimplementation decision.

Figure 2. Components of End-of-Trial Decision-Making in ePCTs

Venn diagram showing where post-trial responsibilities, sustainment, and de-implementation overlap in end-of-trial decision-making.

 

 

 

 

 

 

 

 

 

 

Next Section

SECTIONS

CHAPTER SECTIONS

sections

  1. Introduction
  2. End-of-Trial Decision-Making Challenges
  3. Considerations for Investigators
  4. Recommendations

Resources

Post-trial Responsibilities in Pragmatic Clinical Trials: Fulfilling the Promise of Research to Drive Real-world Change
This journal article, produced by members of the Ethics and Regulatory and Implementation Science Cores, explores the responsibilities that occur at the conclusion of a pragmatic clinical trial.

Factors Affecting Post-trial Sustainment or De-implementation of Study Interventions: A Narrative Review
This journal article, written by investigators from 6 of the NIH Collaboratory's completed pragmatic clinical trials, summarizes the posttrial interpretation of their trials' results and considers the factors that influenced the sustainment or de-implementation of the studies' interventions.

REFERENCES

back to top

Cho HL, Danis M, Grady C. 2018. Post-trial responsibilities beyond post-trial access. Lancet. 391(10129):1478-1479. doi: 10.1016/S0140-6736(18)30761-X. PMID: 29676274.

Miller CJ, Barnett ML, Baumann AA, et al. 2021. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 16(1):36. doi: 10.1186/s13012-021-01105-3. PMID: 33827716.

Morain S, Largent E. 2023. Think Pragmatically: Investigators' Obligations to Patient-Subjects When Research is Embedded in Care. Am J Bioeth. (8):10-21. doi: 10.1080/15265161.2002.2063435. PMID: 35435790.

Morain S, O'Rourke PP, Ali J, et al. 2024. Post-trial responsibilities in pragmatic clinical trials: Fulfilling the promise of research to drive real-world change. Learn Health Syst. 8(3):e10413. doi: 10.1002/lrh2.10413. PMID: 39036536.

Palinkas LA., Chou CP, Spear SE, et al. 2020. Measurement of sustainment of prevention programs and initiatives: the sustainment measurement system scale. Implementation Sci. doi:10.1186/s13012-020-01030-x. PMID: 32883352.

back to top

Proctor E, Silmere H, Raghavan R, et al. 2011. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 38(2):65-76. doi: 10.1007/s10488-010-0319-7. PMID: 20957426.

Scheirer MA, Dearing JW. 2011. An agenda for research on the sustainability of public health programs. Am J Public Health. 101(11):2059-67. doi: 10.2105/AJPH.2011.300193. PMID: 21940916.

Walsh-Bailey C, Tsai E, Tabak RG et al. 2021. A scoping review of de-implementation frameworks and models. Implementation Sci. doi:10.1186/s13012-021-01173-5. PMID: 34819122.


Version History

Published May 9, 2025

current section :

Introduction

  1. Introduction
  2. End-of-Trial Decision-Making Challenges
  3. Considerations for Investigators
  4. Recommendations

Citation:

End-of-Trial Decision-Making: Introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Pragmatic Trials Collaboratory. Available at: https://rethinkingclinicaltrials.org/chapters/dissemination/end-of-trial-decision-making/introduction-end-of-trial-decision-making/. Updated May 15, 2025.

Footer Menu

  • How to Use This Site
  • About NIH Collaboratory
  • Enrollment Reporting
  • Grand Rounds
  • Funding Statement
Link to Twitter Link to LinkedIn Link to Blue Sky Link to NIH Collaboratory email

Reference in this Web site to any specific commercial products, process, service, manufacturer, or company does not constitute its endorsement or recommendation by the U.S. Government or National Institutes of Health (NIH). NIH is not responsible for the contents of any “off-site” Web page referenced from this server.

Log in
Privacy Statement
WordPress is a content management system and should not be used to upload any PHI as it is not an environment for which we exercise oversight, meaning you the author are responsible for the content you post. Please use this system accordingly. Site Map