August 11, 2025: New Living Textbook Chapter Explores Implementation in Pragmatic Clinical Trials

The NIH Pragmatic Trials Collaboratory Implementation Science Core, led by Devon Check and Hayden Bosworth, has developed a new chapter on implementation to assist study teams with the complex process of using and studying implementation strategies to help implement research findings into clinical care. The chapter includes sections on:

Case studies are used to illustrate how pragmatic clinical trials embedded in healthcare systems use implementation frameworks, including examples from RAMP, BEST-ICU, STOP CRC, TSOS, ABATE, STEP-2, and GRACE.

For more, see our collection of chapters on Dissemination and Implementation, which includes chapters on Dissemination to Different Stakeholders, Data Sharing and Embedded Research, and End-of-Trial Decision-Making.

December 12, 2024: A Year of Innovations and Insights From the NIH Pragmatic Trials Collaboratory

A graphic showing a collection of journal covers.In 2024, experts from the NIH Pragmatic Trials Collaboratory published the results of newly completed studies, shared insights from program leadership, and developed innovative methods in the design, conduct, and analysis of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the Core Working Groups, and results from the NIH Collaboratory Trials.

The program contributed more than 30 articles to the peer-reviewed literature this year, including the primary results of the ICD-Pieces and Nudge trials. Several cross-Core and cross-Trial collaborations led to the sharing of important lessons from the conduct of multiple NIH Collaboratory Trials.

The total number of published articles from the program surpassed 340.

Coordinating Center

Cross-Core and Cross-Trial Collaborations

Core Working Groups

Biostatistics and Study Design Core

Electronic Health Records Core

Ethics and Regulatory Core

Community Health Improvement Core

Implementation Science Core

Patient-Centered Outcomes Core

NIH Collaboratory Trials

ABATE Infection

BackInAction

BeatPain Utah

EMBED

FM-TIPS

GGC4H

GRACE

I CAN DO Surgical ACP

ICD-Pieces

LIRE

NOHARM

Nudge

OPTIMUM

PRIM-ER

PROVEN

SPOT

STOP CRC

TSOS

Ensuring Participant Representativeness: STOP CRC

Ensuring Participant Representativeness: STOP CRC

Description

The STOP CRC trial investigators deployed many strategies to ensure the inclusion of all populations in their trial, including piloting the trial, refining the materials based on patient feedback, assembling an advisory board, and conducting plan-do-study-act cycles at the sites. This video describes their experiences with building partnerships throughout this process.

Biography

Amanda Petrik, PhD
Kaiser Permanente Center for Health Research

Gloria Coronado, PhD
Kaiser Permanente Center for Health Research
STOP CRC NIH Collaboratory Trial co-PI

Related

Participant Recruitment

View the full Grand Rounds video

March 30, 2022: Two Weights Make a Wrong: New Article From the Biostatistics and Study Design Core

Contemporary Clinical TrialslsIn a new article from the NIH Pragmatic Trials Collaboratory Biostatistics and Study Design Core, the authors share analytic considerations for cluster randomized trials with hierarchical nesting of participants within clusters. The authors illustrate the problem using theoretical derivations, a simulation study, and data from the STOP CRC NIH Collaboratory Trial as an example.

“We conclude that an analysis using both an exchangeable working correlation matrix and weighting by inverse cluster size, which may be considered the natural analytic approach, can lead to incorrect results. That is, two weights make a wrong. The bias is minimal when there is homogeneity of treatment effects according to cluster size but unacceptable when there is heterogeneity of treatment effects according to cluster size. In addition, we show that only an analysis with an independence working correlation matrix and weighting by inverse cluster size always provides valid results for the UATE [unit average treatment effect] estimand.”

Read the full article.

June 4, 2021: Inclusion of Diverse Participants in Pragmatic Clinical Trials: Planning for Diversity – Stakeholder Engagement and Site Selection to Maximize Diversity (Wendy Weber, ND, PhD, MPH; Julie Fritz, PhD, PT; David Wetter, PhD, MS; Gloria Coronado, PhD; Amanda Petrik, MS)

Speakers

Panelists:

BeatPain Utah  
Julie Fritz, PhD, PT
Associate Dean for Research, College of Health
Adjunct Professor, Orthopaedic Surgery
Distinguished Professor, Physical Therapy & Athletic Training
Adjunct Professor, Orthopaedic Surgery Operations
University of Utah  

David Wetter, PhD, MS
Professor, Population Health Sciences
Adjunct Professor, Psychology
University of Utah  

STOP CRC  
Gloria Coronado, PhD
Distinguished Investigator Mitch Greenlick Endowed Scientist for Health Disparities
Kaiser Permanente Center for Health Research  

Amanda Petrik, MS
Sr. Research Associate Center for Health Research
Kaiser Permanente Northwest

Guest Moderator:

Wendy Weber, ND, PhD, MPH
Chief, Clinical Research in Complementary and Integrative Health Branch
Division of Extramural Research
National Center for Complementary and Integrative Health (NCCIH)
National Institutes of Health (NIH)

Topic

Inclusion of Diverse Participants in Pragmatic Clinical Trials: Planning for Diversity – Stakeholder Engagement and Site Selection to Maximize Diversity

Keywords

Participant diversity; Stakeholder engagement strategies; STOP CRC; BeatPain Utah; Health equity; Community health centers; Plan-Do-Study-Act

Key Points

  • Disparities in pain prevalence and pain management are well established. In particular, odds of receiving opioids for pain management are greater, and odds of nonpharmacologic care lower, in rural, low income, and Latinx communities.
  • It is essential that communities and researchers come together to create long-term solutions to prevent cancer, chronic and infectious disease, and improve health among underserved populations. Research projects should be driven by the priorities of community partners.
  • Study advisory boards can include health center leaders, patient advocates, legislators, and community organization leaders. These boards provide local context and, in the case of STOP CRC, they identified policy changes that were needed around access to colorectal cancer screening.

Discussion Themes

Sharing information and resources with community partners—and responding to their needs and building their capacity—helps to build trust around medicine and research.

To what extent is it important to show evidence of interest in and respect for the community beyond the specific focus of your initiative? For example, participating in important community activities and developing a deeper understanding of the culture.

We must design and plan for sustainability at the outset, and we must make targeted, specific efforts to ensure the inclusion of diverse populations in clinical trials.

Another way to increase diversity is to prepare and train more scientists and investigators from diverse populations.

Read more about the BeatPain Utah and STOP CRC Demonstration Projects.

Tags

#pctGR, @Collaboratory1

Grand Rounds Diversity Workshop Series June 4: Inclusion of Diverse Participants in Pragmatic Clinical Trials: Planning for Diversity – Stakeholder Engagement and Site Selection to Maximize Diversity

Guest Moderator:
Wendy Weber, ND, PhD, MPH
Chief, Clinical Research in Complementary and Integrative Health Branch
Division of Extramural Research
National Center for Complementary and Integrative Health (NCCIH)
National Institutes of Health (NIH)

Panel:
BeatPain Utah

Julie Fritz, PhD, PT
Associate Dean for Research, College of Health
Adjunct Professor, Orthopaedic Surgery
Distinguished Professor, Physical Therapy & Athletic Training
Adjunct Professor, Orthopaedic Surgery Operations
University of Utah

David Wetter, PhD, MS
Professor, Population Health Sciences
Adjunct Professor, Psychology
University of Utah

STOP CRC

Gloria Coronado, PhD
Distinguished Investigator
Mitch Greenlick Endowed Scientist for Health Disparities
Kaiser Permanente Center for Health Research

Amanda Petrik, MS
Sr. Research Associate
Center for Health Research
Kaiser Permanente Northwest

Topic: Inclusion of Diverse Participants in Pragmatic Clinical Trials: Planning for Diversity – Stakeholder Engagement and Site Selection to Maximize Diversity
Date: Friday, June 4, 2021, 1:00-2:00 p.m. ET

Meeting Info: To check whether you have the appropriate players installed for UCF (Universal Communications Format) rich media files, go to https://dukemed.webex.com/dukemed/systemdiagnosis.php.

To join the online meeting:
Go to https://dukemed.webex.com/dukemed/j.php?MTID=m228b52a4666c1e24046e11c01ae7bd0d

You must log in to the URL first.
Click ‘Audio Conference’
Choose ‘I will call in’, select the Toll number.
Dial in using the information from the dialog box that appears.
Be certain to use the Access Code AND the Attendee ID.

Troubleshooting:
If the URL above does not work, go to dukemed.webex.com and enter:
Meeting Number: 120 909 9614
Meeting Password: 1234

For Audio ONLY:
Call-in toll number (US/Canada): 1-650-479-3207
Access code: 120 909 9614

NOTE: For Toll-free users, the call-back (call me) services are also available.

December 15, 2020: A Year of Results and New Insights From the NIH Collaboratory

Collection of Journal CoversNIH Collaboratory researchers in 2020 reported study results, generated new knowledge, and developed innovative research methods in pragmatic clinical trials. Their work included insights from the Coordinating Center and Core Working Groups, analyses from the NIH Collaboratory Distributed Research Network, and results and methodological approaches from the NIH Collaboratory Trials.

So far this year, the NIH Collaboratory has produced more than 3 dozen articles in the peer-reviewed literature, including the primary results of the PROVEN and LIRE trials, the study design of ACP PEACE, insights into the COVID-19 pandemic from TSOS and EMBED, and more:

NIH Collaboratory Coordinating Center

NIH Collaboratory Distributed Research Network

ACP PEACE NIH Collaboratory Trial

EMBED NIH Collaboratory Trial

HiLo NIH Collaboratory Trial

LIRE NIH Collaboratory Trial

PPACT NIH Collaboratory Trial

PRIM-ER NIH Collaboratory Trial

PROVEN NIH Collaboratory Trial

SPOT NIH Collaboratory Trial

STOP CRC NIH Collaboratory Trial

TSOS NIH Collaboratory Trial

July 14, 2020: PIs of Completed NIH Collaboratory Trials Share Accomplishments

In 2012, the NIH Common Fund established the NIH Health Care Systems Research Collaboratory. The goal of the program is to improve the way clinical trials are conducted by creating an infrastructure for collaborative research with healthcare systems. The NIH Collaboratory launched with a Coordinating Center, Core Working Groups, and NIH Collaboratory Trials to conduct embedded pragmatic clinical trials (ePCTs) in partnership with healthcare system leaders and to work collaboratively with the NIH to solve problems as they arise, develop best practices, and share lessons and resources to with others conducting ePCTs.

Collaboratory Mission: Strengthen the national capacity to implement cost-effective large-scale research studies that engage healthcare delivery organizations as research partners.

With the first round of NIH Collaboratory Trials nearing completion, the project teams are beginning to publish results and share lessons with other researchers. We asked the principal investigators of the most recently completed projects to share insights about the important contributions of their studies.

Congratulations on finishing your NIH Collaboratory Trial: What do you think is the most important contribution of your study?

ABATE was conducted to determine whether routine bathing and showering with chlorhexidine soap would reduce multidrug-resistant organisms and bloodstream infections compared with usual care. The trial was conducted in 53 HCA Healthcare hospitals (194 non–critical care units) and included 340,000 patients in the intervention period.

Picture of Dr. Susan Huang
Dr. Susan Huang, Active Bathing to Eliminate (ABATE) Infection

“We found that there was no overall benefit to universal antiseptic bathing in non–intensive care units (ICUs). This is in stark contrast to the huge benefit demonstrated in ICUs in the REDUCE-MRSA trial, and may reflect the fact that non–critical care patients stay only a few days in the hospital and are less likely to develop infection. Nevertheless, we did find that antiseptic bathing and nasal decolonization reduced bloodstream infections and antibiotic-resistant organisms by over 30% in patients with devices outside of the ICU. This is important because they are 10% of the non-ICU population, but responsible for over half of bloodstream infections. They provide a valuable targeted population who appear to benefit from this intervention.”

LIRE was conducted to test the effectiveness of a simple and inexpensive intervention: inserting epidemiologic benchmarks into lumbar spine imaging reports. The goal of the trial was to reduce subsequent tests and treatments, including cross-sectional imaging (such as magnetic resonance imaging and computed tomography), opioid prescriptions, spinal injections, or surgery.

 

Dr. Jeffrey Jarvik, Lumbar Imaging with Reporting of Epidemiology (LIRE)

“I think that one of the most important contributions of the LIRE trial was demonstrating the feasibility of randomizing hundreds of thousands of patients to receive or not receive an intervention that we inserted into the radiology report. Before our trial began, there was a fair amount of skepticism about whether radiologists would accept routinely inserting prevalence information into their reports on a wide scale. We showed without a doubt that it was feasible.”

PPACT was designed to assess the potential benefit of helping patients adopt self-management skills for chronic pain, limit use of opioid medications, and identify factors amenable to treatment in the primary care setting in three Kaiser Permanente (Northwest, Georgia, and Hawaii) involving approximately 800 patients.

Dr. Lynn Debar, Collaborative Care for Chronic Pain in Primary Care (PPACT)

“We started a trial when everybody was still uncertain about what the trade-offs between external validity (and real-world issues that are important for implementation) and the rigor of internal validity. I don’t know if we got that right. There was an assumption that the trial needed to be cluster randomized, and I think it’s informative that only 1 of the 11 NIH-DOD-VA Pain Management Collaboratory trials was cluster randomized. We needed to be able to incubate, have embedded teams stay over time, and really shift the culture. Patients needed to get used to the idea of non-pharmacotherapy over several months, and we may have had more success if we had individually randomized our cohort. I learned a lot in this process.”

PROVEN was designed to evaluate the effectiveness of advance care planning video tools in the nursing home setting by partnering with 2 large healthcare systems that operate 492 nursing homes nationwide.

Dr. Susan Mitchell, Pragmatic Trial of Video Education in Nursing Homes (PROVEN)

“PROVEN found an ACP Video Program did not significantly impact hospital transfers, burdensome treatments, or hospice enrollment among nursing home residents with advanced illness, however intervention fidelity was low. Nonetheless, PROVEN was one of the first large pragmatic trials conducted in US nursing homes. Thus, I feel its greatest contribution was setting a foundation of knowledge for the field in terms of methodologies that enable pragmatic trials in this setting and challenges to overcome.”

STOP CRC was conducted to determine whether EHR-embedded tools and clinic staff training in how to implement a mailed fecal immunochemical test (FIT) outreach program could increase colorectal cancer screening uptake among patients with historically lower CRC screening rates and worse CRC outcomes, such as those with low income, or who are on Medicaid or underinsured. STOP CRC was conducted in 26 Federally Qualified Health Centers (FQHCs) in Oregon and California and involved approximately 41,000 patients.

Picture of Dr. Beverly Green
Dr. Beverly Green, Strategies and Opportunities to Stop Colorectal Cancer (STOP CRC)

“The ability to work with FQHCs and their new electronic data systems was an important contribution. FQHC settings are not organized healthcare systems, such as Kaiser Permanente, where research is more routine. I think we contributed to the success of this type of research and enabled the FHQCs’ ability to do more of it.” — Dr. Beverly Green

Picture of Dr. Gloria Coronado
Dr. Gloria Coronado, Strategies and Opportunities to Stop Colorectal Cancer (STOP CRC)

“Our study designed real-time electronic health record tools to allow clinics to mail cancer screening tests to adults who were overdue. We learned a lot about the challenges clinics faced in implementing the program. We shared our learnings with hundreds of additional community clinics in Washington, Oregon, California to help them anticipate and overcome these challenges.”— Dr. Gloria Coronado

TiME was conducted to determine whether treatment with hemodialysis sessions that are longer than many patients in the United States currently receive reduces the high rate of mortality among people being treated with thrice-weekly maintenance hemodialysis. The trial was conducted in 2 large US dialysis provider organizations, DaVita, Inc. and Fresenius Medical Care – North America, and included 266 outpatient dialysis facilities with 7035 patients.

Picture of Dr. Laura Dember
Dr. Laura Dember, Time to Reduce Mortality in End-Stage Renal Disease (TiME)

“TiME established a model for conducting real-world research for a group of patients for whom there is very little clinical trial data. Many of the approaches and lessons from TiME are now being applied to a new set of pragmatic trials in dialysis that are being conducted in the US and internationally. In my view, TiME’s greatest contribution was to create a foundation for ongoing efficient and rigorous evidence generation in dialysis.”

Data and resources from the NIH Collaboratory Trials are posted on the NIH Collaboratory’s Data and Resource Sharing page in the coming months. As part of the program’s commitment to sharing, all NIH Collaboratory Trials are expected to share data and resources, such as protocols, consent documents, public use datasets, computable phenotypes, and analytic code.

Primary Outcome Papers for TiME, ABATE, STOP CRC, PROVEN

December 18, 2019: NIH Collaboratory Shares New Findings and Fresh Insights in 2019

NIH Collaboratory researchers in 2019 continued to generate new knowledge and research methods in pragmatic clinical trials. Their work included insights from the Coordinating Center and Core Working Groups, large-scale analyses of data from the NIH Collaboratory Distributed Research Network, and results and innovative methodological approaches from the NIH Collaboratory Trials.

So far this year, the NIH Collaboratory has produced nearly 3 dozen articles in the peer-reviewed literature, including the primary results of the ABATE Infection trial, confirmation by the TiME trial of the feasibility of embedding large pragmatic trials in clinical care, and more:

NIH Collaboratory Coordinating Center

NIH Collaboratory Distributed Research Network

ABATE Infection NIH Collaboratory Trial

EMBED NIH Collaboratory Trial

PPACT NIH Collaboratory Trial

PRIM-ER NIH Collaboratory Trial

PROVEN NIH Collaboratory Trial

SPOT NIH Collaboratory Trial

STOP CRC NIH Collaboratory Trial

TiME NIH Collaboratory Trial

TSOS NIH Collaboratory Trial

May 2, 2019: Cost-Effectiveness Analysis of STOP CRC Trial Finds Wide Variation Across Health Centers

A cost-effectiveness analysis of the Strategies and Opportunities to Stop Colorectal Cancer in Priority Populations (STOP CRC) trial, an NIH Collaboratory Trial, revealed wide variation across participating health centers. The study’s findings reflect the complexity of implementing an intervention in pragmatic research involving community health clinics with diverse patient populations, clinic structures, and resources.

The study was published recently in Preventive Medicine.

The STOP CRC trial tested a program to improve colorectal cancer screening rates in 26 clinics within 8 federal qualified health centers in California and Oregon. Intervention clinics embedded a tool in the electronic health record to identify patients who were overdue for screening and mailed a fecal immunochemical test (FIT) kit to these patients. Screening rates were higher overall in intervention clinics than in clinics that practiced usual care, despite low and highly variable rates of implementation of the program among participating clinics.

In the subsequent cost-effectiveness study, variability in program implementation was likewise a key factor in the results. Intervention delivery costs were highly variable across health centers, and the incremental cost-effectiveness of the intervention was diminished somewhat because usual care clinics generated more colonoscopies than intervention clinics after abnormal FIT results.

The study has implications for the design of implementation strategies in pragmatic trials that are embedded in diverse community health centers.

The STOP CRC trial was supported within the NIH Collaboratory by a cooperative agreement from the National Cancer Institute and received logistical and technical support from the NIH Collaboratory Coordinating Center. Download a study snapshot of the STOP CRC trial, and learn more about the NIH Collaboratory Trials.