Using electronic health record (EHR) data for research is fundamentally different than collecting the research data prospectively, as is traditional for controlled clinical trials. Several features of EHR systems create these important differences, most importantly being the lack of investigator control over data collection and recording processes in health care facilities. Other factors include the lack of standard definitions for identifying patient cohorts and study-specific outcomes, the challenges associated with completeness of longitudinal data, and potential errors in linkage of records across systems (Zozus et al. 2015). All of these challenge investigators to assure and demonstrate that data are of adequate quality to support research conclusions. While many of the issues addressed in this chapter apply to a broad range of study designs that might use data from the EHR, this chapter describes the use cases and associated challenges for using EHR data in pragmatic clinical trials, particularly those that include randomization. Specifically, we will discuss:

• Prerequisites for conducting pragmatic research using EHR systems
• Developing and refining the research question and defining the data that are essential and necessary to answer that question
• Data sources for explanatory trials vs PCTs
• The role of data as a partial representation of (or surrogate for) clinical phenomena under investigation
• Considerations for the use of EHR data, including understanding bias and provenance, completeness and other dimensions of data quality, and methods for linking between multiple data sources

Challenges and Prerequisites for Using EHR Systems

In a NIH Pragmatic Trials Collaboratory manuscript, 20 NIH Collaboratory Trials responded to a survey about the challenges they encountered when using EHR systems for pragmatic clinical research (Richesson et al. 2021). The goal of the study was to elucidate challenges and develop solutions—or prerequisites for pragmatic research—to enable healthcare system leaders, policy makers, and EHR designers to improve the national capacity for generating real-world evidence. The table summarizes 6 broad challenges and solutions, identified by the study’s authors. The solutions for each broad challenge—if implemented as part of the health systems and research infrastructure—can enable the rapid conduct of future pragmatic trials, and hence can be conceptualized as prerequisites for successful EHR-based pragmatic research.

This study highlights the need to tailor the use of EHR systems to enable the collection of patient-centered outcomes and the extraction of high-quality, standardized data. Although EHR data systems are designed to support clinical care and billing, high-quality data derived from these systems can also help improve population health by generating reliable evidence and advancing continuous learning within and across healthcare systems.

For further descriptions of the 6 challenges and prerequisites, read Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory.

Data Sources for Explanatory Trials vs PCTs

There is a marked contrast between using the data collected within an EHR system for research versus using data that were collected outside of an EHR explicitly for a trial. Traditionally in clinical research, a study protocol specifies the data to be collected, and they are collected through a separate, stand-alone system. The circumstances around data collection for traditional trials, including procedures for taking samples, making observations and recording data (e.g., patient positioning, timing, and anatomical location) are clearly defined in the protocol and the data are collected in accordance with those specifications. Further, in traditional research, the protocol defines the timing of data relative to the trial milestones or activities, for example, “the second assessment occurs 14 days post baseline.” In designing traditional (or explanatory) research studies, a top-down approach is usually taken starting with the research question and working down to the required data.

In contrast, the use of existing data streams, a defining feature for pragmatic clinical trials, presents a number of issues and requires a different approach than in traditional explanatory trials. Data contained in EHRs captured from routine-care settings or insurance claims have a different context from prospectively collected research data. While the context of care and data collection is often unspecified, it is certainly not defined around a research question or protocol. Consequently, the structure and representation of clinical data is imposed at the facility according to their standards for clinical documentation and business needs rather than by the needs of the research study. This structure, along with local context, record linkage considerations, use of diagnosis or other structured codes, etc, brings substantial and unique challenges for using data from EHR systems in research.