December 15, 2025: A Year of Trial Results and Innovations From the NIH Pragmatic Trials Collaboratory

A collage of journal covers with the label "NIH Pragmatic Trials Collaboratory 2025 Publications Roundup"In 2025, NIH Pragmatic Trials Collaboratory investigators published new study designs and trial results, shared insights from program leadership, and developed innovative methods in the design, conduct, implementation, and dissemination of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the Core Working Groups, and results from the NIH Collaboratory Trials.

The program contributed 45 articles to the peer-reviewed literature this year, including the primary results of the ACP PEACE, BackInAction, HiLo, INSPIRE, and PRIM‑ER trials. Cross-Core and cross-Trial collaborations led to the sharing of important lessons from the conduct of multiple NIH Collaboratory Trials.

The total number of published articles from the program reached 386.

Coordinating Center

Cross-Core and Cross-Trial Collaborations

Distributed Research Network

Core Working Groups

Biostatistics and Study Design Core

Community Health Improvement Core

Electronic Health Records Core

Ethics and Regulatory Core

Health Care Systems Interactions Core

Patient-Centered Outcomes Core

NIH Collaboratory Trials

ABATE Infection

ACP PEACE

ARBOR-Telehealth

BackInAction

BeatPain Utah

BEST-ICU

EMBED

FM-TIPS

GGC4H

GRACE

HiLo

I CAN DO Surgical ACP

IMPACt-LBP

INSPIRE

iPATH

LIRE

MOMs Chat & Care Study

NOHARM

Nudge

OPTIMUM

PRIM-ER

SPOT

TAICHIKNEE

June 23, 2025: How to Choose Patient-Reported Outcome Measures in Pragmatic Clinical Trials?

Living Textbook iconA new section of the Living Textbook of Pragmatic Clinical Trials describes considerations for choosing patient-reported outcome measures in pragmatic clinical trials.

“Where possible, investigators are encouraged to use measures with adequate support for validity that are in the public domain,” the authors wrote.

The authors provide a set of questions to guide investigators in choosing appropriate measures. For example, investigators may want to understand whether the patient-reported outcome is in electronic health records, is in the public domain, and is valid for the use case in question.

The considerations were developed by members of the NIH Pragmatic Trials Collaboratory’s Patient-Centered Outcomes Core in collaboration with the Health Care Systems Interactions Core, the Electronic Health Records Core, and colleagues at the NIH.

May 28, 2025: New NIH Collaboratory Learning Module Explores Challenges and Possibilities of Working With Electronic Health Record Data

The NIH Pragmatic Trials Collaboratory has launched a new learning module, Healthcare Data Interoperability and Standardization for Research, exploring the complexities of collecting, storing, and transforming healthcare data in the electronic health record to achieve optimal patient and research outcomes.

The learning module, which was developed by members of the NIH Collaboratory’s Electronic Health Records Core Working Group, comprises 2 new learning topic videos, “The Big Picture: Healthcare Data and Interoperability” and “Common Data Models.” The module is led by Lesley Curtis of Duke University, a co⁠–⁠principal investigator of the NIH Collaboratory Coordinating Center.

“Research requires a common data structure that can be meaningfully compared across sources,” Curtis explained. “To be useful beyond patient care, complex and variable healthcare data must be organized and standardized,” she said.

The first video in the module, “The Big Picture: Healthcare Data and Interoperability,” covers the key concepts of the Fast Healthcare Interoperability Resources (FHIR) standard, the US Core Data for Interoperability (USCDI) standard, and the Trusted Exchange Framework and Common Agreement (TEFCA). The module explains that these key elements of healthcare data collection, storage, and transfer are a step toward greater interoperability for the US healthcare system.

“Common Data Models” explores the tools and processes available for transforming healthcare data into research data. The video introduces common data models such as Sentinel, OMOP, and PCORnet and explains how each model accomplishes the goal of transforming raw data into a standard format, curating this data for accuracy and completeness, and phenotyping for consistency.

“By employing frameworks that improve healthcare data interoperability and using tools to standardize data structure, we move closer to unlocking the full potential of healthcare data,” Curtis said.

Find all the NIH Collaboratory Learning Modules.

December 12, 2024: A Year of Innovations and Insights From the NIH Pragmatic Trials Collaboratory

A graphic showing a collection of journal covers.In 2024, experts from the NIH Pragmatic Trials Collaboratory published the results of newly completed studies, shared insights from program leadership, and developed innovative methods in the design, conduct, and analysis of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the Core Working Groups, and results from the NIH Collaboratory Trials.

The program contributed more than 30 articles to the peer-reviewed literature this year, including the primary results of the ICD-Pieces and Nudge trials. Several cross-Core and cross-Trial collaborations led to the sharing of important lessons from the conduct of multiple NIH Collaboratory Trials.

The total number of published articles from the program surpassed 340.

Coordinating Center

Cross-Core and Cross-Trial Collaborations

Core Working Groups

Biostatistics and Study Design Core

Electronic Health Records Core

Ethics and Regulatory Core

Community Health Improvement Core

Implementation Science Core

Patient-Centered Outcomes Core

NIH Collaboratory Trials

ABATE Infection

BackInAction

BeatPain Utah

EMBED

FM-TIPS

GGC4H

GRACE

I CAN DO Surgical ACP

ICD-Pieces

LIRE

NOHARM

Nudge

OPTIMUM

PRIM-ER

PROVEN

SPOT

STOP CRC

TSOS

August 13, 2024: EHR Core and Patient-Centered Outcomes Core Reflect on Challenges in Pragmatic Trials

At the NIH Pragmatic Clinical Trials Collaboratory’s 2024 Annual Steering Committee Meeting, leaders of the Electronic Health Records (EHR) Core and the Patient-Centered Outcomes (PCO) Core discussed challenges and lessons learned in the NIH Collaboratory Trials.

EHR cochairs Keith Marsolo and Rachel Richesson shared challenges related to EHR usage, including EHRs switching midtrial, unanticipated changes to the EHR that impacted intervention delivery, and difficulties navigating data sharing policies.

“We continue to be challenged, but we fall back on our core principles, which is looking for good quality data and good quality research with an eye on how we can share the results in the underlying data in the future,” Richesson said.

Over the next year, the EHR Core will explore improving data access, transparency of computable phenotypes, and capturing dietary supplement information in the EHR.

PCO Core cochairs Cristy Zigler and Emily O’Brien shared goals for effective collection PCO data.

“We have advised studies about the importance of delivering patient-reported outcomes to participants in a way that is easy to complete, doesn’t overburden them, and is relevant for their population,” O’Brien said.

The Core has compiled a list of best practices for collecting PCO data and collaborated with the Health Equity Core to create a Health Equity Toolkit, designed to give investigators practical guidance for incorporating health and equity considerations in research.

The Core has also been working closely with Implementation Science Core to consider the sustainability of PCO data systems and facilitate reuse of collection methods.

“We often see separate systems set up to collect PCO data for the trial, and then those systems sunset at the end and are not in place to support an assessment over time,” O’Brien said.

Access the complete meeting materials from the 2024 Annual Steering Committee Meeting.

July 10, 2023: EHR Core’s Keith Marsolo Shares Scientific Goals and Resources Needed For Data Sharing

In an interview at the NIH Pragmatic Trials Collaboratory Steering Committee Meeting in May, Keith Marsolo, PhD, Co-Chair of the Electronic Health Records Core, reflected on data sharing experiences from the program and factors affecting the meaningful re-use of data.

Headshot of Dr. Keith Marsolo
Keith Marsolo, PhD

“The data sharing experiences from the NIH Pragmatic Trials Collaboratory Trials have been fairly limited. Most studies have only had one or two requests for data. I think part of the impetus for the new NIH Data Management and Sharing Policy is to spur additional re-use of research data, ” Marsolo said. He elaborated that the scientific goals for data sharing include transparency, reproducibility, validation, new generative science, and respecting the contribution of participants.

Data that can be shared include scientific data (raw data, analytic dataset, etc) and metadata (protocol, analytic code, statistical analysis plan, etc). “If we are going to share data, we want it to be useful. For new science, you likely need the raw data. For reproducibility, you need a lot of types of data. Study teams can be more upfront about the data and metadata that can be shared and the use cases they can support,” Marsolo said.

Because data from pragmatic trials are collected as part of routine care, there are restrictions as the data contain not only personal health data but also data about the healthcare system, so what can actually be shared may be limited. Overall, Marsolo suggests that more information is needed about what data can be shared and how this translates into the goals for re-use.

“I think if we’re not clear about what limitations of the data are, there could be a mismatch between expectations of what we can get from these data and what’s actually achievable,” Marsolo said.

With regards to resources needed for data sharing, Marsolo stated, “There might be funding that needs to be allocated to promote data sharing. For example, PCORI [The Patient-Centered Outcomes Research Institute] has a model where they allocate funding specifically for data sharing and dissemination.”

The EHR Core can help to promote data sharing by working to provide examples on how to navigate data sharing and outlining different approaches for pragmatic trials. NIH can provide additional guidance on how to handle datasets with restrictions—what to share, for what purpose, and at what cost.

For more information, Marsolo and colleagues published an article on data sharing in which the authors suggest that data sharing is not rising to its potential, and that more guidance is needed to prevent data sharing from becoming a “box-checking exercise.” There is also a Living Textbook chapter on Data Sharing and Embedded Research.

All of the materials from the 2023 Steering Committee meeting are now available.

July 3, 2023: Report Shares Strategies for Addressing Lack of Generalizability of EHR Data

JAMIA cover imagePragmatic research is vulnerable to differences in data capture and access to care for different subsets of the population, which, if left unaddressed, can worsen health gaps.

In a new article published online ahead of print in JAMIA, the NIH Pragmatic Trials Collaboratory's Trial teams reflect on the challenges encountered by their trials and share the specific strategies they used to increase the generalizability of research results.

“Poor generalizability can occur because detailed information about specific populations is missing, and critically, is missing not at random,” the authors cautioned.

The NIH Collaboratory Trials are implementing approaches designed to ensure inclusion and retention of all populations, and to enable the complete collection of data.

“By improving data capture, access to care, and patient technology support, ePCTs hold the potential to yield insights and estimates pertinent to the entire population, not just a subset of the population,” they wrote.

This work was a collaboration between the Community Health Improvement Core, the EHR Core, and the Patient-Centered Outcomes Core of the NIH Pragmatic Trials Collaboratory.

July 14, 2022: Solving Unexpected Challenges in Pragmatic Trial Data Collection

Headshot of Dr. Keith MarsoloDuring the NIH Pragmatic Trials Collaboratory Steering Committee meeting and 10th anniversary celebration, Dr. Keith Marsolo, Co-chair of the Electronic Health Records Core, moderated a panel on “Barriers and Challenges: Data Collection and Merging Datasets.” Four panelists shared their experiences collecting and aggregating data from diverse data sets and how they solved unexpected challenges.

The panelists included Drs. Ted Melnick (EMBED), Holt Oliver (ICD-Pieces), Margaret Kuklinski (GGC4H), and Andrea Cheville (NOHARM).

During the discussion, panelists shared lessons learned from collecting electronic health record (EHR) data during large pragmatic clinical trials across multiple health systems. Common challenges included:

  • Privacy concerns and not having the ability to collect deidentified patient or clinician demographic information
  • Working across health systems that use different EHR platforms and collecting data consistently across sites
  • Turnover of IT staff at participating sites and changes in site data agencies during the study period
  • Updates to the EHR that affect study algorithms and other issues with algorithms not identifying data as expected
  • Overcoming technical barriers with practice workflows and integrating with IT systems

The panelists shared solutions and possible best practices for future studies, including the need for planning, coordinating, and testing before study launch, the importance of being able to pivot and change directions as problems arise, being open to alternative data collection methods such as surveys to augment findings, and having the right team at the right time to be responsive to problems, which for one study meant having informaticists and expert EHR builders embedded in the trial team.

Panelists identified two key policy and infrastructure changes that would help trials be successful in the future. These changes include development of a national unique patient identifier across health systems and incentivizing EHR vendors to find common ground to better support research.

The Electronic Health Records Core continues to learn from experiences of the program’s NIH Collaboratory Trials and shares emerging information, resources, and EHR-related recommendations to improve future pragmatic research.

View slides from the discussion panel.

Learn more

View video collection that highlights advances in Electronic Health Records for pragmatic research.

Watch the August 2021 Interview with Electronic Health Record Core leaders Drs. Rachel Richesson and Keith Marsolo

August 11, 2021: EHR Core Facing Familiar Challenges, Intensified by Pandemic

Leaders of the NIH Collaboratory’s Electronic Health Records (EHR) Core Working Group spoke in a recent interview about the impacts of the COVID-19 pandemic on pragmatic trials. They also talked about upcoming projects and a recent survey of the NIH Collaboratory Trials.

 

“The pandemic amplified themes that we’ve heard all along, which is how to get resources, how to get support for the data that we need for these trials or to configure EHRs as we need for these trials,” said Dr. Rachel Richesson, a professor of learning health sciences at the University of Michigan and a cochair of the EHR Core. “The pandemic just shifted priorities tremendously and made it quite challenging. As a result, we’ve had discussions about…really making the value case for pragmatic research and embedded research,” Richesson said.

View the full video.

Dr. Keith Marsolo, also a cochair of the Core, added, “The pandemic obviously was a big challenge for health systems in general as they transitioned to telehealth, dealt with shutdowns, things of that nature.” Marsolo is an associate professor in population health sciences at Duke University.

“Things are starting to move forward a little bit more. In one [NIH Collaboratory Trial], they were able to leverage some infrastructure that was purchased to help provide telehealth services for their region. So they’ve been able to bootstrap their trial off of that infrastructure,” Marsolo said.

Richesson also described an upcoming paper reporting the results of a recent survey of the NIH Collaboratory Trials. The survey focused on the challenges of using EHR data in pragmatic trials embedded in healthcare systems.

“No surprise on the challenges we encountered: It’s still challenging to get data from organizations for research, the data are still heterogeneous,…and there are challenges on the research team to ensure that that information is equivalent and how these data can be pulled together to support the research question,” Richesson said. “There’s a particular emphasis with the newer studies on patient-reported outcomes and how to get those collected as part of routine care,” she said.

View the full video.

 

Screen shot of video interview with Dr. Rachel Richesson and Dr. Keith Marsolo
Dr. Rachel Richesson and Dr. Keith Marsolo