Implementation Science Core

February 24, 2026: Pragmatic Clinical Trials Offer Unique Opportunities to Study Implementation Processes and Outcomes

Posted on by Karen Staman

The interventions tested in pragmatic clinical trials are embedded into routine care practices, which offers opportunities to study implementation and sustainment outcomes.

In an article published online ahead of print in Implementation Science Communications, members of the NIH Pragmatic Trials Collaboratory’s Implementation Science Core share the results of a survey about the measurement of implementation outcomes in pragmatic trials.

Of the 32 NIH Collaboratory Trials, 29 completed the survey (91% response rate).

Main findings:

Across the trials, measurement was high for some constructs, including:

  • Reach (91%)
  • Adherence (76%)
  • Fidelity (93%)
  • Adaptations (69%)

However, evaluation for some constructs remains low:

  • Costs (31%)
  • Clinician adoption and representativeness (45%)
  • Anticipated sustainment (24%)
  • Actual sustainment (38%)

The authors suggest there is opportunity for growth in measuring some of these important implementation constructs.

“To benefit from the substantial investment into pragmatic clinical trials, we need to improve measurement of constructs that drive the implementation of evidence into routine care, including information about costs, sustainability, and sustainment,” the authors wrote.

Measurement of these constructs in future pragmatic trials could result in development of improved implementation strategies to increase the likelihood of effective implementation leading to equitable, sustainable, and scalable improvement in practice.

Read the full article.

For more information about implementation in pragmatic clinical trials, see the Living Textbook Chapter on Implementation.

December 12, 2024: A Year of Innovations and Insights From the NIH Pragmatic Trials Collaboratory

Posted on by Damon Seils

A graphic showing a collection of journal covers.In 2024, experts from the NIH Pragmatic Trials Collaboratory published the results of newly completed studies, shared insights from program leadership, and developed innovative methods in the design, conduct, and analysis of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the Core Working Groups, and results from the NIH Collaboratory Trials.

The program contributed more than 30 articles to the peer-reviewed literature this year, including the primary results of the ICD-Pieces and Nudge trials. Several cross-Core and cross-Trial collaborations led to the sharing of important lessons from the conduct of multiple NIH Collaboratory Trials.

The total number of published articles from the program surpassed 340.

Coordinating Center

Cross-Core and Cross-Trial Collaborations

Core Working Groups

Biostatistics and Study Design Core

Electronic Health Records Core

Ethics and Regulatory Core

Community Health Improvement Core

Implementation Science Core

Patient-Centered Outcomes Core

NIH Collaboratory Trials

ABATE Infection

BackInAction

BeatPain Utah

EMBED

FM-TIPS

GGC4H

GRACE

I CAN DO Surgical ACP

ICD-Pieces

LIRE

NOHARM

Nudge

OPTIMUM

PRIM-ER

PROVEN

SPOT

STOP CRC

TSOS

January 22, 2024: Your Pragmatic Trial Has Ended. Now What?

Posted on by Damon Seils

What happens to a pragmatic trial intervention after the study ends?

In a new article from the NIH Pragmatic Trials Collaboratory, investigators from 6 of the program's completed pragmatic clinical trials summarize the posttrial interpretation of their trials' results and consider the factors that influenced the sustainment or de-implementation of the studies' interventions.

The article was published online ahead of print this month in the Journal of General Internal Medicine.

Three lessons emerged from the review:

  • Ineffective interventions may be sustained if they have other measured or perceived benefits.
  • Effective interventions may not be sustained if they require significant resources.
  • Alignment with policy incentives is essential for achieving sustainment and scale-up of effective interventions.

Based on the experiences of the 6 case studies, the authors offer several recommendations to assist pragmatic trials researchers in considering the posttrial sustainment or de-implementation of their study interventions.

Read the full article.

This work was produced by members of the NIH Collaboratory's Implementation Science Core. The case studies in the article are from the ABATE Infection, LIRE, PPACT, PROVEN, STOP-CRC, and TSOS trials. Learn more about the NIH Collaboratory Trials.

October 9, 2023: Registration Opens for Pragmatic Trials Workshop at AcademyHealth Dissemination & Implementation Conference

Posted on by Damon Seils

The NIH Pragmatic Trials Collaboratory will offer a full-day workshop at the 16th Annual Conference on the Science of Dissemination and Implementation in Health in Arlington, Virginia. The workshop, “Dissemination & Implementation in Embedded Pragmatic Trials: Raising the Bar for Real-World Research,” will introduce concepts in the design, conduct, and implementation of pragmatic clinical trials embedded in healthcare systems, with a particular focus on methods relevant to health services researchers.

The learning objectives of the workshop include:

  • To identify key areas of synergy between pragmatic trials and implementation research
  • To introduce attendees to the unique characteristics and challenges of designing, conducting, and implementing pragmatic clinical trials embedded within diverse health care systems, and to describe opportunities for integrating implementation research methods into pragmatic trials
  • To increase the capacity of health services researchers to address important clinical questions with embedded pragmatic clinical trials and share lessons from implementation science for supporting intervention adoption, sustainment, scale-up, and/or deimplementation

The theme of this year’s D&I conference is “Raising Expectations for D&I Science: Challenges and Opportunities.” The annual conference is cohosted by the NIH and AcademyHealth.

WORKSHOP DETAILS AND REGISTRATION
Sunday, December 10, 10:00 am-6:00 pm
Gateway Marriott, Arlington, Virginia

August 7, 2023: Implementation Science Core Cochairs Discuss Implementation Challenges, Strategies in Pragmatic Trials

Posted on by Hannah Webster
Headshots of Dr. Devon Check and Dr. Hayden Bosworth
From left: Devon Check and Hayden Bosworth

In an interview at the annual NIH Pragmatic Trials Collaboratory Steering Committee meeting, Implementation Science Core cochairs Hayden Bosworth and Devon Check reflected on the Core’s goals and factors for implementation success.

The Implementation Science Core, which launched in 2022, supports implementation-related research aims in pragmatic clinical trials to promote the uptake and sustainability of effective interventions in routine practice.

While the Core is still in its early days, the cochairs have plans for how to best serve the NIH Collaboratory. One of their first initiatives is to develop standardized measurements for evaluating implementation. They are currently gaining consensus around those measurements among the Core’s members. 

“We are in the phase of starting to understand the challenges related to implementation during the trial and then afterwards,” Bosworth said. 

Stakeholder Engagement Early and Often 

Check highlighted early, multilevel stakeholder engagement as a critical step to promoting implementation success.

“The buy-in that you get to conduct a trial can be very different than the buy-in necessary to stand an intervention up in practice,” she said. 

Several NIH Collaboratory Trial representatives have highlighted logistics challenges across large health systems as a reoccurring issue.

“Even an intervention that is effective, and potentially also cost-effective, might not be sustained by health systems because it’s just not feasible with the existing resources,” Check said. 

It can also be valuable to engage policymakers that can promote implementation.

“Early and ongoing engagement of policymakers who are in the position to make policy changes or requirements based on study findings can sometimes overcome health systems barriers,” she said. 

Prioritizing Implementation in Study Design 

Bosworth discussed the value of making implementation a priority from the very beginning.

“Are there ways that we can start thinking about implementation from the study design stage?” he said.  “Even if it is a terrific study with great findings, if it’s not feasible or too complex to implement, it’s frustrating to wait to consider implementation.”   

Engaging with staff can provide insight on the many complexities of multi-level health systems. The study design stage of a pragmatic clinical trial should be seen as an opportunity to promote future implementation, he said. 

A Safe Space for Both Failures and Successes 

The cochairs highlighted the importance of creating an environment where project teams can share their challenges and failures just as openly as their successes.

“For everything that works, there are probably 10 things that have gone wrong,” Bosworth said. “If we don’t report that, we do the same things over and over. It’s just as valuable to hear what didn’t work so that we can move forward.” 

Within the Core, they hope to promote a culture of knowledge sharing as well as collecting and sharing the lessons learned among projects.

“We’re looking forward to consulting, collaborating, and being available as problems and successes arise,” Bosworth said. “And we want to share those lessons learned.” 

“Documenting and collecting data on what the challenges are is a useful early step, so that we can characterize the challenges and think about solutions and recommendations for pragmatic trials.” Check said.

August 3, 2023: NCI’s David Chambers Discusses Role of Implementation Science in Pragmatic Research

Posted on by Gina Uhlenbrauck

This year’s Annual Steering Committee Meeting for the NIH Pragmatic Trials Collaboratory featured implementation science as one of the topics of focus. In an interview after his keynote presentation, Dr. David Chambers, deputy director for implementation science at the National Cancer Institute, shared his thoughts on opportunities for implementation science in the context of pragmatic trials.

Why Implementation Science Is Important in Pragmatic Research

Chambers described the overlap between implementation science and pragmatic research and how this creates several benefits. Implementation science gives a heightened focus on how to get interventions to be as accessible, well used, and beneficial as possible to populations within the systems and communities in which people are seeking and receiving healthcare.

“The earlier researchers can think about their interventions being used beyond the trial, the better,” Chambers explained. “Implementation science helps to provide a lens for multiple levels of change that may be needed, including supports for patients and families, clinicians, clinics, systems, policymakers, and other key decision-makers.”

Furthermore, implementation science methods can save time in identifying barriers and facilitators for delivering interventions with high quality-knowledge which can be applied toward the ultimate use of interventions.

Chambers encourages researchers to embrace the dynamism that is reflected in our health systems, which is particularly apt for research conducted in the setting of routine care. “It is a given that there will be deviations from the design of an intervention and its implementation, so how can researchers learn from this?” he said.

In terms of sustaining an intervention, a more dynamic approach is needed to how the intervention and the context will change over time, he explained. “Too often we think of sustainment in terms of fixing things in their original state. Sustainment needs to think about how to build in evolution-medicine and our practices are evolving.”

Role of the NIH Pragmatic Trials Collaboratory

According to Chambers, the NIH Pragmatic Trials Collaboratory provides a natural setting for implementation activities because of the close partnerships between the investigative teams and the health systems and community settings where the research occurs. As a result, the program is poised for ongoing learning from the strategies health systems are using to implement a range of different interventions.

There is also an opportunity for pooling lessons across the NIH Collaboratory Trials, such as common measures that can be used to better characterize adaptation, understanding perspectives and needs of patients and clinicians, and approaches related to long-term sustainment or deimplementation of interventions.

Increasingly for NIH Collaboratory trials, issues related to implementation are baked into both the design of interventions and the approach used to test the interventions, which is a testament to the program’s ongoing progress in encouraging the use of implementation science concepts and methods.

Learn more about the NIH Collaboratory’s Implementation Science Core.

How NIH Is Advancing Implementation Science

For over 20 years, NIH has recognized the gaps in knowledge needed for successful implementation of evidence-based interventions. Across NIH institutes, centers, and offices, they have discussed the common challenges observed when investigators saw a positive result in their trial and were frustrated that it could not be replicated and scaled up in the real world.

NIH funds dissemination and implementation research grants and has a standing review panel, the Science of Implementation in Health and Healthcare, focused on this area where any applicant to NIH can suggest that their grant be reviewed. In addition, NIH supports training opportunities in the field and cohosts an annual scientific conference in partnership with AcademyHealth.

NIH continues to explore the interface between effectiveness and implementation, think about deimplementation, and work on tackling misinformation, all towards advancing how to better apply the evidence generated in research to optimize population health and healthcare.

July 17, 2023: NIH Pragmatic Trials Collaboratory Welcomes First Class of Fellows

Posted on by Elizabeth Mccamic

Headshots of Dr. Stephanie Ibemere and Dr. Kaitlyn McLeod
From left: Dr. Stephanie Ibemere and Dr. Kaitlyn McLeod

The NIH Pragmatic Trials Collaboratory is welcoming its first class of fellows in a new program for early-career investigators with a scholarly interest in pragmatic clinical trials.

"We are very excited to welcome the 2 inaugural fellows to the NIH Pragmatic Trials Collaboratory Fellowship Program and give them the education and tools that they need to launch their careers as pragmatic clinical trials investigators," said Dr. Rosa Gonzalez-Guarda, cochair of the NIH Collaboratory’s Fellowship Program and Health Equity Core.

Dr. Stephanie Ibemere is an assistant professor in the Duke University School of Nursing. Dr. Ibemere will work closely with the Implementation Science Core and the GRACE NIH Collaboratory Trial team during her fellowship year.

Dr. Kaitlyn McLeod is a resident physician in the Internal Medicine Residency Program at the University of Colorado. Dr. McLeod will work closely with the Community Health Improvement Core and the Nudge NIH Collaboratory Trial team during her fellowship year.

"We are delighted that Stephanie and Kaitlyn are joining the NIH Collaboratory as fellows, and we look forward to the insights they will bring to the Cores and NIH Collaboratory Trials," said Dr. Kevin Weinfurt, co–principal investigator of the Coordinating Center and cochair of the new Fellowship Program.

During the 1-year fellowship, which began July 1, 2023, fellows will become members of the NIH Collaboratory program and have access to a wide range of activities and opportunities. Fellows will be active participants in a Core Working Group that aligns with their interests and will also be given education on the fundamentals of designing and conducting successful pragmatic clinical trials.

Working with a NIH Collaboratory Trial investigator mentor and a Core leadership mentor from the NIH Collaboratory, each fellow will complete a project that contributes to the field of pragmatic trials, resulting in a new publishable product or resource. In addition, fellows will participate in the following activities:

  • Regular presentations and discussions with experienced pragmatic trial investigators and leaders in the field
  • Individualized coursework and mentorship
  • Guided research program
  • Professional development training
  • Presentation and publication opportunities
  • Training on health equity as it relates to pragmatic trials

Ibemere and McLeod joined leaders from the NIH Collaboratory and their mentors for a welcome and orientation meeting on July 6 to kick off their fellowship year.

January 30, 2023: NIH Pragmatic Trials Collaboratory Begins 2023 With New Cores in Health Equity and Implementation Science

Posted on by Damon Seils

Headshots of Dr. Rosa Gonzalez-Guarda and Dr. Cherise Harrington
Dr. Rosa Gonzalez-Guarda (left) and Dr. Cherise Harrington (right)

The NIH Pragmatic Trials Collaboratory began 2023 with 2 new Core Working Groups focused on health equity and implementation science. The membership of both Cores draws from across the program to support the design and implementation of the NIH Collaboratory Trials.

“By launching these 2 new Core Working Groups, the NIH Pragmatic Trials Collaboratory is reaffirming its commitment to generating knowledge that supports equitable pragmatic research to improve healthcare for all,” said Dr. Adrian Hernandez, co-principal investigator of the NIH Pragmatic Trials Coordinating Center.

The Health Equity Core provides leadership and guidance to help the NIH Collaboratory Trials and investigators be more equitable in research. The Core’s work includes supporting pragmatic trials to address social and structural drivers of inequities, implement patient and community engagement strategies, promote the inclusion and mentorship of historically underrepresented scientists, and develop culturally and contextually aligned research and translation strategies that overcome bias and resonate with patients and communities.

The Health Equity Core is led by Dr. Rosa Gonzalez-Guarda and Dr. Cherise Harrington. Gonzalez-Guarda is an associate professor of nursing at Duke University, faculty lead for the Population Health Research Area of Excellence at Duke’s Center for Nursing Research, and codirector of the Community Engagement Core for Duke Clinical Translational Science Institute. Harrington is a senior researcher and associate professor of public health education at North Carolina Central University. Alex Fist of the Duke Clinical Research Institute serves as the Core’s project manager in the Coordinating Center. The Core’s members include Jessica Lee Barnhill, Sheana Bull, Gaby Castro, Andrea Cheville, Allison Cuthel, Dana Dailey, Juanita Darby, Graham Dore, Julie Fritz, Morgan Fuoco, Christine Goertz, Katharine Lawrence, Vivian Lyons, Keith Marsolo, Alice Pressman, Nina Siman, and Miguel Vazquez.

Headshots of Dr. Devon Check and Dr. Hayden Bosworth
Dr. Devon Check (left) and Dr. Hayden Bosworth (right)

The Implementation Science Core provides technical support and pragmatic trial expertise for NIH Collaboratory Trials with a specific focus on innovative dissemination and implementation science approaches. The Core will study methods and strategies to promote the uptake of interventions that have proven effective in routine practice, with the aim of improving population health.

The Implementation Science Core is co-led by Dr. Devon Check and Dr. Hayden Bosworth. Check is an assistant professor in population health sciences at Duke University. Bosworth is a professor in population health sciences, psychiatry, and nursing at Duke and deputy director for the Center for Health Services Research in Primary Care at the Durham VA Medical Center. Jill George of the Duke Clinical Research Institute serves as the Core’s project manager in the Coordinating Center. The Core’s members include Kristin Archer, Gaby Castro, Allison Cuthel, Ardith Doorenbos, Carol Greco, Crystal Patil, Sarah Redmond, Isabel Jordan Roth, Stacie Salsbury, Stacy Sterling, Anne Thackeray, Cindy Tofthagen, Katy Trinkley, Miguel Vazquez, and Angelo Volandes.

“Through their work in the areas of implementation science and health equity, the new Cores can help uncover how best to implement improved clinical practices so the benefits reach at-risk and traditionally underserved populations,” Hernandez said.