Although ePCT interventions are often considered to be “static” products, which helps ensure the fidelity of what is being delivered across sites, when interventions involve the EHR, they are anything but static. The reality is that ePCT interventions are implemented within a dynamic, operational EHR that primarily exists to serve the healthcare enterprise.

This leads to 3 challenges for those designing and conducting ePCTs (Marsolo et al 2024):

• Rapid technology change over the lifespan of a trial (typically several years) can have unexpected consequences.
• EHR updates can cause system changes that affect intervention delivery.
• Switching EHR systems among health systems can create new complexities.

Several investigators of the NIH Collaboratory Trials shared how these challenges affected their research, along with steps they took to minimize the impact of EHR changes and maintain the validity of their trials:

Case Example: NOHARM

In the NOHARM trial, a routine upgrade to the institutional EHR rendered the study algorithm responsible for automatically assigning trial patient-reported outcome measures (PROMs) dysfunctional. The algorithm determined when a trial participant underwent a qualifying surgical procedure and used this criterion as the basis for initiating a series of post-surgical PROMs assignments.  The failure to assign post-surgical PROMs went undetected until it was serendipitously noted several months later.  However, by that time, key data collection opportunities had been missed. The missed data collection points posed a significant threat to the trial’s internal validity. In response, the team developed algorithm validation processes that they now run monthly and with each Epic upgrade. Proactively, the team is also working with the Mayo Clinic Enterprise EHR group to identify the elements of future updates that might degrade NOHARM algorithms and clinical decision support tools.  The research team was able to recoup some of the missing PROM assessments by utilizing score collected from participants during the same time frame for clinical applications.  For example, the NOHARM trial’s pain and function PROMs were routinely administered pre- and post-surgery by several participating surgical departments (Redmond et al. 2022).  Use of condition agnostic, commonly used PROMs proved to be a tremendous benefit in this regard.

Case Example: EMBED

In EMBED, investigators built the trial intervention locally at the first site to create a streamlined workflow in the EHR. The goal of the intervention was to help clinicians initiate buprenorphine in the emergency department in response to the opioid crisis. However, even in the first year, workflows changed and the EHR vendors started offering more integrated pathways for initiating buprenorphine as clinical decision support. The EMBED team could have used these same pathways if they had started their trial 6 months later. Once other pathway options became available, the team allowed each health system to use what was most accepted at the time in their health system as long as there was fidelity with what the intervention offered (Grudzen et al. 2019).

Case Example: LIRE

For LIRE, the EHR was used to deliver a clinical decision support intervention, which involved insertion of age-appropriate prevalence data for common imaging findings (Jarvik et al. 2020). The intervention was implemented differently across different sites: one site inserted pop-up alerts for clinicians, which were not permanently attached to the report. Another site inserted the intervention using the radiology reporting system, which allowed the dictating radiologist to remove intervention text if desired. Thus, the team had to monitor implementation of interventions by using regular data pulls. This led to the fortunate discovery that an upgrade to the dictation software at one of the sites had broken the link that inserted the intervention text. As a result, the frequency of monitoring was increased (Curtis et al. 2025).

Case Example: SPOT

In 2015, the transition from ICD-9 to ICD-10 codes occurred, and SPOT investigators noticed substantial changes in coding of “intent” for injuries and poisonings during that same time period. This likely represented artifacts of the coding transition rather than true changes in suicidal behavior. The study team also found that the changes in coding practices were different across sites, and one site had their own proprietary set of codes (Stewart et al. 2017).

Recommendations

EHRs are updated at a regular cadence and offer new modules that can change or replace existing workflows. Based on these experiences, NIH Collaboratory investigators recommend that trials plan for regular fidelity checks and budget for IT personnel to support these checks for the duration of the trial. Conducting repeated, careful, and rigorous checks to detect possible changes can ensure the intervention is being delivered as intended (Curtis et al. 2025).

Additionally, engaging with partners can help ensure that the intervention is not deployed concurrently with the release of new functionality.