This phase involves evaluating the context, capabilities, and challenges of the partner healthcare system and testing key elements of the intervention and collection and transfer of data from the EHR. Feasibility assessment specific to embedded PCTs may be associated with the following challenges.
Study teams should work early on with their statistician to anticipate gaps or issues around cluster randomization, including sample size and potential for contamination, intraclass correlation, varying cluster size, the need for stratification or matching, and potential for missing follow-up data.
The Collaboratory’s Biostatistics and Study Design Core provides resources to help address challenges related to:
- Statistical lessons learned
- Constrained randomization
- Small-sample robust variance correction
- Frailty models
- Unequal cluster sizes
- Pair-matching versus stratification
- Intraclass correlation coefficient
- Extracting usable data from EHRs
Secondary Use of EHR Data
Using EHR data for research is fundamentally different from using prospectively collected data. Several aspects of EHR data drive these differences, including the lack of control over data definitions and data collection processes in healthcare facilities, procedures for access to the data, frequent dependence on record linkage, the need for computable definitions for cohorts and outcomes of interest, and the intricacies of demonstrating that data are of adequate quality to support research conclusions.
Consider how the intervention will use existing EHR data for cohort identification, recruitment, sample size estimates, population screening, collection of embedded patient-reported outcome (PRO) data, and so on. If there will be different EHR systems, how will they be linked? Pilot test the data collection procedures and any web-based tools developed specifically for the study.
Capabilities and Readiness of the Partner Healthcare System
Consider whether there is a gap in existing services for the target population, or whether the healthcare system has had difficulties successfully addressing patient needs. It is important to evaluate how the embedded intervention will align with the goals of the healthcare system to achieve their practice metrics. Also consider the best timing for embedding the PCT intervention if there are expected EHR platform changes or pending external contextual or policy changes that could impact the system’s capacity to implement the PCT.
Consider testing the intervention across partner systems that vary in capacity or context; for example, a VA health system and a regional or community health system. Evaluate how effective the system or clinic will be as a research partner. One way is for the study team to pilot the programming capacity of a site (e.g., giving the site sample programming code, or asking sites to send the code that they will use to the study coordinating center) to determine if the site is able to fully participate. Researchers could devise a set of criteria for site inclusion. For example, in a trial in which epidemiological benchmarks are inserted into radiology imaging reports, the study team wanted to verify at each site that the intervention text could be successfully included in the report based on a specific CPT code, modality, patient age, and date. Another way to gauge the effectiveness of a partnership is to confirm the healthcare system’s commitment to identifying the clinical staff available to carry out the study intervention.
Integrating the Study Into the Clinical Workflow
Test how smoothly the embedded intervention will be incorporated into the site’s existing system infrastructure. Sites have different workflows and available resources, which can complicate the process. In pragmatic research, it is important that study procedures mimic routine practice. Consider how individual healthcare providers will be engaged in the intervention and how burdensome it may be for them. Make an effort to simplify and streamline the study to make integration as user-friendly as possible. Be open to adapting the study based on stakeholder input or changes in the care delivery system. Study teams are also encouraged to pilot the effectiveness of trial-related materials such as informed consent forms and processes, training videos, brochures and placards, and toolkits.
- What are 2-5 aspects of your trial that are essential to pilot?
- Developing the Trial Documentation
- Establishing Close Partnerships With Participating Healthcare System Leaders and Staff
- Delineating the Roles of All Stakeholders to Determine Training Needs
- Pilot Testing
- Feasibility Assessment Scenarios From the Collaboratory’s Demonstration Projects
- Spotlight on Four Demonstration Projects
- Additional Resources
Diffusion of innovations in service organizations: systematic review and recommendations (Greenhalgh T, et al. Milbank Quarterly 2004)
A guide to research partnerships for pragmatic clinical trials (Johnson KE, et al. BMJ 2014).
Trials without tribulations: minimizing the burden of pragmatic research on healthcare systems (Larson EB, et al. Healthcare 2015).