Emily O’Brien

October 21, 2025: New Podcast Episode Points Investigators to Strategies for “Avoiding the Fumble” in Pragmatic Trials

Posted on by Sophia Ramirez

In a new episode of the Rethinking Clinical Trials Podcast, Emily O’Brien of Duke University expanded on key takeaways from her recent Grand Rounds presentation, “Avoiding the Fumble: Building on a Decade of Lessons from Pragmatic Clinical Trials.”

Listen to the podcast. For alerts about new episodes, subscribe for free on Spotify, Amazon Music, Apple Podcasts, or SoundCloud.

The Grand Rounds presentation came on the heels of a big decade for pragmatic clinical trials, defined by new opportunities and challenges.

“We’ve seen a huge explosion in interest,” said O’Brien. “And that often comes some reality checks as people begin to understand what it takes to do one of these studies well across multiple sites.”

“The concept of the fumble has been an interesting way to acknowledge that we don’t always get things perfectly right the first time.”

O’Brien and her colleagues on the PCORnet® team developed “The Playbook” as a tool for sharing and refining the best approach to national-scale research. It is a pragmatic application of principles often discussed within the NIH Collaboratory, providing investigators with practical strategies for implementation.

Like the Living Textbook, the Playbook is a living document that will be updated to reflect learnings from the successes and fumbles of future trials.

“If we can establish a culture of transparency and sharing, that can accelerate the implementation of best practices and refinement of those practices,” O’Brien said.

O’Brien is an associate professor in population health sciences at Duke University. She serves as cochair of the NIH Collaboratory’s Patient-Centered Outcomes Core Working Group.

Podcast October 15, 2025: “Avoiding the Fumble” in Pragmatic Clinical Trials (Emily O’Brien, PhD, FAHA)

Posted on by Sophia Ramirez

In this episode of the NIH Collaboratory podcast, Dr. Emily O’Brien expands on key takeaways from her recent Grand Rounds presentation, “Avoiding the Fumble: Building on a Decade of Lessons from Pragmatic Clinical Trials.”

Read the transcript. For alerts about new episodes, subscribe for free on Spotify, Amazon Music, Apple Podcasts, or SoundCloud.

Grand Rounds August 22, 2025: Avoiding the Fumble: Building on a Decade of Lessons from Pragmatic Clinical Trials (Emily O’Brien, PhD, FAHA)

Posted on by Sophia Ramirez

Speaker

Emily O’Brien, PhD, FAHA
Associate Professor
Duke Clinical Research Institute
Duke University School of Medicine
Department of Population Health Sciences

Keywords

Pragmatic Trials; Best Practices; PCORnet; Evidence-Based Practices

Key Points

  • Historically, the healthcare industry has been limited by an insufficient body of evidence driving everyday clinical decision-making. Roughly a decade ago, pragmatic clinical trials (PCTs) began to gain traction as a promising solution.
  • There are several advantages of PCTs. They can be embedded within healthcare systems without disrupting the clinical workflow; answer questions of major public health importance; streamline procedures and infrastructure by making use of existing data; and include diverse, representative study populations for highly generalizable results.
  • But a recent analysis of clinical research site challenges noted that protocol complexity, site workload, and patient burden have increased since 2015. Though the analysis was not specific to pragmatic trials, a fundamental shift in how researchers think about study design is required across the clinical trials space.
  • Additionally, evidence-based practices – even those that have been stress-tested in PCTs – are not always adopted by health systems. Trial success does not necessarily coincide with system priorities; different audiences, i.e. systems and funders, require different kinds of evidence; and 5- to 10-year studies are misaligned with systems’ 2- to 3-year decision horizons.
  • The NIH Pragmatic Trials Collaboratory philosophy holds that fumbles are part of the game; we can’t improve if we only share wins, and transparency and teamwork has helped this community iterate and improve. Accordingly, the PCORnet team developed “The Playbook,” inspired by the NIH Collaboratory’s Living Textbook, as a tool for sharing and refining the best approach to national-scale research.
  • The Playbook contains practical “drills” for avoiding common fumbles in recruitment, workflow, and outcome capture, and was created using a user-centered design process. They engaged PCORnet groups, partners, and members of the Playbook’s intended audience to inform and guide the content.
  • Modules 1 – 5 of the Playbook, launching this year, will provide an introduction to the network. They include sections on getting started with PCORnet, utilizing the network’s resources, dissemination and implementation expectations for PCORnet studies, and case studies.
  • In the long-term, the PCORnet team plans to actively review, maintain, and expand the Playbook. Additional modules are in process and targeted for release in 2026.

Discussion Themes

The success of the Playbook may depend on the willingness of investigators to share both their “best plays” and their mistakes. Dr. O’Brien was optimistic that research teams will buy into this philosophy and acknowledge it as an important piece of the evidence generation process.

The case studies that the team selected serve to illustrate A) that PCORnet trials are unique, innovative, and approaching challenges in a thoughtful, inspiring way and B) the many ways to engage with the network.

August 20, 2025: PCORnet Playbook Featured in This Week’s Rethinking Clinical Trials Grand Rounds

Posted on by Damon Seils

Emily O'Brien
Dr. Emily O’Brien

In this Friday’s Rethinking Clinical Trials Grand Rounds, Emily O’Brien of Duke University will present “The PCORnet Playbook: Avoiding the Fumble in Pragmatic Trials.”

The Grand Rounds session will be held on Friday, August 22, 2025, at 1:00 pm eastern.

O’Brien is an associate professor in population health sciences at Duke University. She serves as cochair of the NIH Collaboratory’s Patient-Centered Outcomes Core Working Group.

Join the online meeting.

July 31, 2025: Pragmatic Trialists Share Strategies for Monitoring Changes in Usual Care

Posted on by Karen Staman

Pragmatic clinical trials are conducted as part of routine healthcare delivery and often compare an intervention to usual care. To do this, researchers must understand, monitor, and document standard care at participating research sites.

At the NIH Pragmatic Trials Collaboratory’s 2025 Annual Steering Committee Meeting, Duke University’s Emily O’Brian asked a panel of trialists about the strategies they used to define and document usual care. The panelists included Rachel Winer, co–principal investigator (PI) of STEP-2;, Richard Platt, co-PI of INSPIRE; and Christine Goertz, co-PI of IMPACt-LBP.

Key Strategies

  • Establish a community of individuals who are interested in and committed to answering the research question
  • Develop relationships: visit each site and have monthly coaching calls
  • Get commitment in advance from sites to hold their practice constant for the duration of the trial
  • Have sites complete readiness surveys or feasibility assessments that include questions about potential upcoming quality improvement initiatives
  • Minimize burden on sites as much as possible

The Navigating the Unknown chapter of the Living Textbook includes  descriptions of unanticipated challenges that may occur during the years-long course of a study that can have profound effects on usual care, including:

These challenges all require close collaboration with research partners to develop solutions.

This summer, we are sharing highlights from the 2025 Annual Steering Committee Meeting. Access the complete collection of meeting materials.

August 13, 2024: EHR Core and Patient-Centered Outcomes Core Reflect on Challenges in Pragmatic Trials

Posted on by Karen Staman

At the NIH Pragmatic Clinical Trials Collaboratory’s 2024 Annual Steering Committee Meeting, leaders of the Electronic Health Records (EHR) Core and the Patient-Centered Outcomes (PCO) Core discussed challenges and lessons learned in the NIH Collaboratory Trials.

EHR cochairs Keith Marsolo and Rachel Richesson shared challenges related to EHR usage, including EHRs switching midtrial, unanticipated changes to the EHR that impacted intervention delivery, and difficulties navigating data sharing policies.

“We continue to be challenged, but we fall back on our core principles, which is looking for good quality data and good quality research with an eye on how we can share the results in the underlying data in the future,” Richesson said.

Over the next year, the EHR Core will explore improving data access, transparency of computable phenotypes, and capturing dietary supplement information in the EHR.

PCO Core cochairs Cristy Zigler and Emily O’Brien shared goals for effective collection PCO data.

“We have advised studies about the importance of delivering patient-reported outcomes to participants in a way that is easy to complete, doesn’t overburden them, and is relevant for their population,” O’Brien said.

The Core has compiled a list of best practices for collecting PCO data and collaborated with the Health Equity Core to create a Health Equity Toolkit, designed to give investigators practical guidance for incorporating health and equity considerations in research.

The Core has also been working closely with Implementation Science Core to consider the sustainability of PCO data systems and facilitate reuse of collection methods.

“We often see separate systems set up to collect PCO data for the trial, and then those systems sunset at the end and are not in place to support an assessment over time,” O’Brien said.

Access the complete meeting materials from the 2024 Annual Steering Committee Meeting.

November 22, 2022: ADAPTABLE Points to Need for Better Integration of Patient-Reported Health Data Into Pragmatic Trials

Posted on by Damon Seils

Headshot of Dr. Emily O'BrienA concordance analysis from ADAPTABLE, a large pragmatic, comparative effectiveness trial, found low to moderate agreement between patient-reported health data and data derived from the electronic health record (EHR). The findings highlight the need for better integration of patient-reported health data into pragmatic research studies.

Results of the study were published in JAMA Cardiology.

Patient-reported health data can assist investigators in capturing clinical events in pragmatic clinical trials. However, little information is available about the fitness for use of patient-reported data in large-scale health research.

In this study, analysts evaluated the concordance of a set of variables that were both reported by patients through an online portal and available in the EHR as part of the trial. The validity of participant self-report varied by the method of data collection, the participant population, and the type of information collected. For example, patient report of clinical events had low sensitivity compared with EHR data for myocardial infarction, stroke, and major bleeding events. Coronary revascularization was the most concordant event by data source, but agreement was only moderate.

“As the use of direct-to-participant research designs grows, it’s important to understand how participant-reported data lines up with other data types,” said Emily O’Brien, an associate professor in population health sciences at Duke University and the lead author of the study.

“Our findings suggest that some caution is warranted in this space—especially if relying on participant report alone for capture of cardiovascular events—and we need more research on how to best design questions and set up participant portals to maximize validity of this type of data,” O’Brien said.

ADAPTABLE, the first major randomized comparative effectiveness trial conducted by the National Patient-Centered Clinical Research Network (PCORnet), seeks to determine the optimal dose of aspirin therapy for secondary prevention of atherosclerotic cardiovascular disease. The trial relies on both existing EHR data sources and patient-reported health data captured through an online portal.

This is the first study to assess validity of participant-reported health data collected through an online portal. The study’s results are consistent with prior evidence suggesting that the validity of participant-reported events varies by data collection modality, participant subgroup, and the type of information collected.

This work was supported by a supplemental grant award to the NIH Pragmatic Trials Collaboratory Coordinating Center from the National Center for Complementary and Integrative Health.

Read the full report in JAMA Cardiology.

October 26, 2022: In This Friday’s PCT Grand Rounds, an Update on the HERO Registry

Posted on by Damon Seils

Headshots of Dr. Emily O'Brien an Dr. Russell RothmanIn this Friday’s PCT Grand Rounds, Emily O’Brien of Duke University and Russell Rothman of Vanderbilt University will present “The HERO (Healthcare Worker Exposure Response & Outcomes) Program: An Online Community to Support Observational Studies, Randomized Trials, and Long-Term Safety Surveillance.” The Grand Rounds session will be held on Friday, October 28, 2022, at 1:00 pm eastern.

O’Brien is an associate professor in population health sciences at Duke University and a cochair of the NIH Collaboratory’s Patient-Centered Outcomes Core. Rothman is senior vice president for population and public health and director of the Vanderbilt Institute for Medicine and Public Health at Vanderbilt University Medical Center.

Join the online meeting.

June 29, 2022: Pragmatic Trials Experienced Significant Impacts of COVID-19 on Trial Activities

Posted on by Damon Seils

Headshot of Dr. Emily O'Brien
Lead author Dr. Emily O’Brien

Pragmatic clinical trials experienced significant impacts of COVID-19 on trial activities, according to a survey of NIH Pragmatic Trials Collaboratory Trials published last month in Trials.

Since the beginning of the COVID-19 pandemic in March 2020, many of the NIH Collaboratory Trials have had to postpone recruitment, alter methods of participant engagement, and modify tools for research assessment and intervention delivery. Trial delays often created statistical challenges, especially for those using stepped-wedge designs.

Most of the study teams were able to work closely with healthcare system partners to adapt their study interventions. Some of the adaptations resulted in better participant outreach and the ability to study virtual intervention delivery methods. All of the NIH Collaboratory Trials were able to adapt to pandemic-related changes.

In a recent video interview, lead author Dr. Emily O’Brien praised the study teams.

“We certainly did see some degree of disruption, but I was impressed by the flexibility in the ability to adapt that all of our NIH Collaboratory Trial teams showed,” said O’Brien. “Some of them mentioned the support of having the [Core Working Groups] and consultations with experts in real time to be able to make adjustments,” she said.

The NIH Collaboratory published a tool last year to assist investigators in identifying impacts of the COVID-19 pandemic on ongoing pragmatic trials. The Statistical Analysis Plan Checklist for Addressing COVID-19 Impacts summarizes impacts on trial conduct that study teams should document, measure, analyze, and report.

May 10, 2022: Pragmatic Trials Learn From Disruption in the COVID-19 Pandemic

Posted on by Damon Seils

In an interview at the NIH Pragmatic Trials Collaboratory Steering Committee’s annual meeting in April, Dr. Emily O’Brien and Dr. Adrian Hernandez discussed challenges for pragmatic trials during the COVID-19 pandemic and lessons for the future of the healthcare system.

“We certainly did see some degree of disruption, but I was impressed by the flexibility in the ability to adapt that all of our NIH Collaboratory Trial teams showed,” said O’Brien. “Some of them mentioned the support of having the Cores and consultations with experts in real time to be able to make adjustments,” she said.

O’Brien is a an associate professor in population health sciences at Duke University and a cochair of the NIH Collaboratory’s Patient-Centered Outcomes Core. Hernandez, the executive director of the Duke Clinical Research Institute, is a co–principal investigator of the program’s Coordinating Center. View the full interview.

O’Brien and her colleagues at the NIH Collaboratory Coordinating Center conducted a brief survey about the impacts of the pandemic on the NIH Collaboratory Trials, such as challenges they encountered, solutions they implemented, and any new measures the study teams started using.

“The impacts on the NIH Collaboratory Trials [included] research staff being redeployed to COVID-19 studies or clinician-scientists moving into predominantly clinical care roles as opposed to research roles,” O’Brien explained. “The most obvious impact of those changes was delays across projects. It took longer to get things done, to enroll patients. That was felt pretty universally by our projects, although there were several projects that were further along, had enrollment mechanisms that were in place and already up and running that were less impacted by those delays,” she said.

Some study teams also reported a renewed commitment from their partnering healthcare systems to work together as they encountered challenges in trial implementation.

Hernandez added, “For many studies, we thought they would actually stop. And for traditional trials, that’s what happened. But for [the NIH Collaboratory Trials], since they’re closer, more embedded within healthcare systems, part of the delivery process, they were able to flex because by nature they have to for any environment that changes.”

When asked what the COVID-19 pandemic has taught about what is needed for the future, Hernandez said, “It’s not too different than, say, what we have with the Defense Production Act…. What we need is essentially the equivalent of that, a ‘Pandemic Response Production Act,’ so that healthcare systems can be organized, coordinated to generate evidence around what’s needed for the response.”