During the course of the years-long pragmatic clinical trials supported by the NIH Pragmatic Trials Collaboratory, many unanticipated challenges have occurred, some of which have had profound effects on usual care, trial implementation, data systems, and staff. These unanticipated changes threatened the ability of the trials to address the questions they were designed to answer. A new chapter of the Living Textbook—Navigating the Unknown—describes these challenges and the responses of the study teams.

The chapter describes 3 general categories of challenges, each meriting a different response:

1. If the challenge is a local or temporary issue (for example, a pandemic temporarily shuts down in-person care, or a partnering health system dissolves or is purchased), but the question is still relevant or important and the trial is still feasible, then a workaround may solve the problem.
2. If the trial is no longer feasible for some reason (for example, the recruitment process is not feasible, or the intervention cannot be delivered as planned), and the question is still relevant, it is necessary to make significant changes to the protocol.
3. If the question is no longer relevant or important (for example, new evidence or policy changes make the question no longer relevant), the trial should not continue. For this challenge, it may necessary either to stop the trial or to make fundamental changes to address a different question (since the original question is no longer relevant).

The chapter describes local or temporary challenges some of the study teams faced, such as the COVID-19 pandemic, health system mergers, and changes to the electronic health record (EHR). In these cases, the research questions were still relevant and important and the trial designs were still feasible, so workarounds were created to solve the problems.

• Section 2: Study teams responded to staff turnover, leadership changes, and health system acquisitions and mergers.
• Section 3: Rapid technology change created unexpected consequences, such as EHR updates causing system changes that affected intervention delivery, and sites switching EHRs systems creating complexities during the trial.
• Section 4: COVID-19 had significant impacts on trial activities.

Section 5 of the new chapter addresses barriers that resulted from aspects of the protocol that could have impacted recruitment, retention, or implementation in a way that imperiled the ability of trials to answer the question posed by a research study. In these scenarios, researchers found it appropriate to change the protocol or research question—to pivot—in order to glean meaningful, actionable evidence.

Sections 6 and 7 describe challenges that can fall into either category 1 or 2, and investigators had to decide how to respond in real time.

• Section 6: Clinical practice guidelines and policies changed due to new evidence from observational studies, small trials, and shifting expert opinion, and therefore, usual care changed.
• Section 7: Quality improvement initiatives were launched to address similar problems, threatening the ability to discern differences between arms of the trial.

The NIH Pragmatic Trials Collaboratory supports pragmatic clinical trials embedded in healthcare systems to test interventions that address urgent public health problems faced by delivery systems. They involve hundreds to thousands of participants and generally include usual care as a control arm. One of the most important lessons learned through the course of these trials is that unexpected change is a given.

For more, see the section on Unanticipated Changes in the Analysis Plan chapter of the Living Textbook.