Implementing and Monitoring the Delivery of an Intervention

Using Electronic Health Record Data in Pragmatic Clinical Trials

Section 6

Implementing and Monitoring the Delivery of an Intervention


Rachel Richesson, MS, PhD, MPH

Richard Platt, MD, MSc

Gregory Simon, MD, MPH

Lesley Curtis, PhD

Reesa Laws, BS

Adrian Hernandez, MD, MSH

Jon Puro, MPA-HA

Doug Zatzick, MD

Erik van Eaton, MD, FACS

Vincent Mor, PhD


Contributing Editor

Karen Staman, MS

In PCTs, EHR systems might be used as a part of the intervention and/or the data might be used to target or apply the intervention. For example, a best practice alert or a checklist could be embedded into the EHR system and evaluated as part of the PCT. Also, EHR data can support the important process of monitoring the fidelity of the intervention during the conduct of the trial. An on-going monitoring plan to ensure that data collection for the intervention is being captured is very important. In the case of multi-site research, investigators will need to determine if the intervention can be applied in a clinically equivalent way.

  • Is it possible to target and assess the intervention across multiple sites?

Because clinical systems will never be able to capture the data elements needed to answer all research questions, standardized approaches may be needed to augment EHR systems with additional data collection and research add-ons (that are equivalent across sites). Researchers can opt to add new data elements to existing EHRs systems with the caveat that organizations move slowly, and many stakeholders (outside of the PCT) will need to approve and support the addition. Other modes for data collection (laptops for web-based forms, etc.) can be considered as well as dedicated research information systems. Sampling might also be performed in order to identify and fill any gaps in data collection. Not only is it a challenge to get modified records designed to document implementation of an intervention into the EHR, but even when leadership allows this, line staff and providers may not actually use the new records to document their activities. For example, in PROVEN, because the documented rate of showing the videos in a partner facility was quite low, a more aggressive approach was undertaken to stimulate staff to show the video.  The approach revealed that, not infrequently, the video had been shown and patients’ advance directives were changed, but neither of these outcomes were documented in the EHR.

For PCTs, the act of randomization might indeed mitigate at least some of the variation in data quality and thereby reduce the impact of the different sources of error mentioned above. However, the assumption that with proper randomization, any and all confounders will be randomly distributed across groups or clusters is not always correct. Additionally, while PCTs by nature strive to collect as few new data as possible, it is often necessary to prospectively collect new process data on whether the individuals are in the control or intervention group, in order to ensure completeness and fidelity to the intervention. Researchers, informaticists, and healthcare administrators often must work together to support the collection of this type of process data that is critical for PCTs (Marsolo 2013; Richesson et al. 2017). Further, process data can allow pragmatic trials to identify and adapt to external forces that threaten integrity of PCT design. The reporting of data related to where the study was done and characteristics of populations may also help readers assess the generalizability of the trial results to other populations (Kahn et al. 2015; Zozus et al. 2014).




back to top

Kahn MG, Brown JS, Chun AT, et al. 2015. Transparent reporting of data quality in distributed data networks. EGEMS (Wash DC). 3:1052. doi:10.13063/2327-9214.1052. PMID: 25992385.

Marsolo K. 2013. Informatics and operations--let’s get integrated. J Am Med Inform Assoc. 20:122–124. doi:10.1136/amiajnl-2012-001194. PMID: 22940670.

Richesson RL, Green BB, Laws R, et al. 2017 Mar 14. Pragmatic (trial) informatics: a perspective from the NIH Health Care Systems Research Collaboratory. J Am Med Inform Assoc. doi:10.1093/jamia/ocx016. PMID: 28340241.

Zozus MN, Hammond WE, Green BB, et al. 2014. Assessing Data Quality for Healthcare Systems Data Used in Clinical Research. Accessed Aug 14, 2017.

Version History

November 30, 2018: Updated text as part of annual update (changes made by K. Staman).

Published August 25, 2017


Richesson R, Platt P, Simon G, et al. Using Electronic Health Record Data in Pragmatic Clinical Trials: Implementing and Monitoring the Delivery of an Intervention. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: Updated January 14, 2019. DOI: 10.28929/035.