The NIH Pragmatic Trials Collaboratory supports embedded pragmatic clinical trials (ePCTs) that test interventions to address urgent public health problems faced by delivery systems. They involve hundreds to thousands of participants and generally include usual care as a control arm. During the course of these years-long trials, many unanticipated challenges may occur that have profound effects on usual care, trial implementation, data systems, and staff. The unanticipated changes may threaten the ability of the ePCTs to address the questions they were designed to answer.

There are three general categories of unexpected challenges, each meriting a different response:

1. If the challenge is a local or temporary issue (e.g. pandemic temporarily shuts down in-person care, this health system dissolves or is purchased), but the question is still relevant/important and the trial design is still feasible, then a workaround may solve the problem.
2. If the trial design is no longer feasible for some reason (e.g., the recruitment process is not feasible, intervention can’t be delivered as originally planned) and the question is still relevant, it is necessary to make significant changes to the protocol
3. The question is no longer relevant or important, thus the trial should not continue (e.g. new evidence or policy changes make the question no longer relevant). Note we do not provide examples of this scenario but want to acknowledge that it does exist. For this challenge, it may necessary either stop the trial or make fundamental changes to address a different question (since the original question is no longer relevant).

For challenge 1, which concerns external or idiosyncratic events that investigators must adapt to, but that are not intrinsically related to the protocol, investigators will need to modify the design or implementation of the trial, working in partnership with health systems to ensure that the trial can still result in information that informs public health.

As we describe in the following sections, these challenges require temporary work-arounds and close collaboration with partners to develop solutions so that the trial can continue.

• Section 2: Responding to Staff Turnover, Leadership Changes, and Health System Acquisition and Mergers.
• Section 3: Rapid technology change created unexpected consequences, such as electronic health record (EHR) updates causing system changes that affected intervention delivery, and sites switching EHRs systems creating complexities mid-trial (Marsolo et al. 2024).
• Section 4: COVID-19 had significant impacts on trial activities (O’Brien et al. 2022)

Challenge 2 represents barriers that result from aspects of the protocol that impact recruitment, retention, or implementation in a way that imperils the ability of trials to answer the question posed by a research study. In these scenarios it may be appropriate to change the protocol or research question—to pivot—in order to glean meaningful, actionable evidence.

In section 5 we describe:

• Recruitment challenges
• Interventions that do not reach the target population
• Implementation strategies that are not achieving the intended goals (lack of fidelity and resulting lack of exposure to the intervention)

Developing a contingency plan is one way to prepare for major foreseeable changes, and these plans can be included in the trial’s clinical protocol. Many trials include monitoring for adherence to the protocol (fidelity) to ensure that the intervention arm receives adequate exposure to the intervention. What is often lacking is a plan to act based on what is learned from monitoring, along with a mechanism that allows for a pivot. The goal of a pivot is to enable the chance of obtaining useful information from the trial that clinicians and health systems leaders can use to improve care or processes.

Finally, some challenges can fall into either category, and investigators must decide how to respond in real-time. Examples of such unexpected events from the NIH Pragmatic Trials Collaboratory Include:

• Clinical practice guidelines and policies changed due to new evidence from observational studies, small trials, and shifting expert opinion, and therefore, usual care changed (Curtis et al. 2019).
• Quality improvement (QI) initiatives were launched to address similar problems, threatening the ability to discern differences between arms of the trial (Tuzzio et al. 2021).

One of the most important lessons learned by researchers in the NIH Pragmatic Trials Collaboratory is that unexpected change is a given. This chapter discusses how NIH Collaboratory investigators addressed unanticipated challenges to ePCTs, and where possible, developed contingency plans to prepare (as best as one is able) for the unknown.