Biostatistics and Study Design Core

December 15, 2025: A Year of Trial Results and Innovations From the NIH Pragmatic Trials Collaboratory

Posted on by Damon Seils

A collage of journal covers with the label "NIH Pragmatic Trials Collaboratory 2025 Publications Roundup"In 2025, NIH Pragmatic Trials Collaboratory investigators published new study designs and trial results, shared insights from program leadership, and developed innovative methods in the design, conduct, implementation, and dissemination of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the Core Working Groups, and results from the NIH Collaboratory Trials.

The program contributed 45 articles to the peer-reviewed literature this year, including the primary results of the ACP PEACE, BackInAction, HiLo, INSPIRE, and PRIM‑ER trials. Cross-Core and cross-Trial collaborations led to the sharing of important lessons from the conduct of multiple NIH Collaboratory Trials.

The total number of published articles from the program reached 386.

Coordinating Center

Cross-Core and Cross-Trial Collaborations

Distributed Research Network

Core Working Groups

Biostatistics and Study Design Core

Community Health Improvement Core

Electronic Health Records Core

Ethics and Regulatory Core

Health Care Systems Interactions Core

Patient-Centered Outcomes Core

NIH Collaboratory Trials

ABATE Infection

ACP PEACE

ARBOR-Telehealth

BackInAction

BeatPain Utah

BEST-ICU

EMBED

FM-TIPS

GGC4H

GRACE

HiLo

I CAN DO Surgical ACP

IMPACt-LBP

INSPIRE

iPATH

LIRE

MOMs Chat & Care Study

NOHARM

Nudge

OPTIMUM

PRIM-ER

SPOT

TAICHIKNEE

September 8, 2025: P Values vs Decision-Maker Perspectives, in a Special Grand Rounds Session on September 26

Posted on by Damon Seils

In a special session of Rethinking Clinical Trials Grand Rounds on September 26, longtime leaders from the NIH Pragmatic Trials Collaboratory will present “Significance in Pragmatic Clinical Trials: P Values vs Decision-Maker Perspectives.”

The Grand Rounds session will be held on Friday, September 26, 2025, at 1:00 pm eastern.

Greg Simon is a senior investigator at the Kaiser Permanente Washington Health Research Institute, a member of the NIH Collaboratory leadership team, and a cochair of the NIH Collaboratory’s Health Care Systems Interactions Core. Susan Huang is a Chancellor’s Professor in the Division of Infectious Diseases at the UC Irvine School of Medicine and the medical director of epidemiology and infection prevention at UCI Health. She was the principal investigator for ABATE Infection, an NIH Collaboratory Trial. Liz Turner is an associate professor of biostatistics and bioinformatics and global health at Duke University and a cochair of the NIH Collaboratory’s Biostatistics and Study Design Core.

Join the online meeting.

 

March 10, 2025: Developing Monitoring Plans Warrants Special Attention in Pragmatic Clinical Trials

Posted on by Karen Staman

Cover of Contemporary Clinical TrialsIn an article published online ahead of print, leaders from the NIH Pragmatic Trials Collaboratory share lessons learned about the importance of independent oversight by a safety office or data and safety monitoring board in pragmatic clinical trials, even for trials deemed to have minimal risk.

Challenges specific to pragmatic trials include:

  • complexity, quality, and timing of a real-world data pipeline, especially in trials with many heterogeneous sites
  • embedding of interventions in clinical workflows, so investigators have less control over treatments or interventions
  • potential for incidental and collateral findings

“We recommend regular, rigorous data quality checks, ongoing monitoring of adherence to interventions, and including someone who is knowledgeable about pragmatic clinical trials and novel research designs in the development of Data and Safety Monitoring Plans and Data and Safety Monitoring Boards,” the authors wrote.

By attempting to reflect real-world conditions, pragmatic trials are conducted in settings that cannot be closely controlled. Therefore, close monitoring is critical for a successful study that produces meaningful results, whether it be by independent monitors or data and safety monitoring boards.

The authors drew on experiences from 7 of the NIH Collaboratory Trials and the expertise of the Coordinating Center, the Ethics and Regulatory Core, the Biostatistics and Study Design Core, and the Health Care Systems Interactions Core.

The article was published in Contemporary Clinical Trials.

December 12, 2024: A Year of Innovations and Insights From the NIH Pragmatic Trials Collaboratory

Posted on by Damon Seils

A graphic showing a collection of journal covers.In 2024, experts from the NIH Pragmatic Trials Collaboratory published the results of newly completed studies, shared insights from program leadership, and developed innovative methods in the design, conduct, and analysis of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the Core Working Groups, and results from the NIH Collaboratory Trials.

The program contributed more than 30 articles to the peer-reviewed literature this year, including the primary results of the ICD-Pieces and Nudge trials. Several cross-Core and cross-Trial collaborations led to the sharing of important lessons from the conduct of multiple NIH Collaboratory Trials.

The total number of published articles from the program surpassed 340.

Coordinating Center

Cross-Core and Cross-Trial Collaborations

Core Working Groups

Biostatistics and Study Design Core

Electronic Health Records Core

Ethics and Regulatory Core

Community Health Improvement Core

Implementation Science Core

Patient-Centered Outcomes Core

NIH Collaboratory Trials

ABATE Infection

BackInAction

BeatPain Utah

EMBED

FM-TIPS

GGC4H

GRACE

I CAN DO Surgical ACP

ICD-Pieces

LIRE

NOHARM

Nudge

OPTIMUM

PRIM-ER

PROVEN

SPOT

STOP CRC

TSOS

October 15, 2024: Case Study Describes a Reassessment of Sample Size in an Ongoing Cluster Randomized Trial

Posted on by Damon Seils

FM-TIPS logoA new case study from the NIH Pragmatic Trials Collaboratory highlights an interim reassessment of sample size during an ongoing cluster randomized trial. The case study was published this week in the Living Textbook of Pragmatic Clinical Trials.

Researchers in cluster randomized trials must account for potential correlation between clusters in the design and analysis of their trial by estimating the intraclass correlation when calculating the target sample size. Often they use preliminary data from the planned enrollment sites to estimate the correlation. However, when preliminary data are unavailable at the time of study design, they may use interim data collected during the trial itself to reassess the trial’s sample size.

The contributors of the case study focus on FM-TIPS, an NIH Collaboratory Trial, to describe an approach to conducting an interim reassessment of sample size in an ongoing trial. Read the full case study.

FM-TIPS is examining whether the addition of transcutaneous electrical nerve stimulation to routine physical therapy improves movement-evoked pain compared with physical therapy alone among patients with fibromyalgia. The trial is supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases through the NIH HEAL Initiative. Learn more about FM-TIPS.

The contributors of the case study include members of the FM-TIPS study team and leaders of the NIH Collaboratory’s Biostatistics and Study Design Core. David-Erick Lafontant is a statistician, Bridget Zimmerman is a clinical professor of biostatistics, and Emine Bayman is an associate professor of biostatistics—all at the University of Iowa. Megan McCabe is an assistant professor of biostatistics at the University of Alabama at Birmingham. Patrick Heagerty is a professor of biostatistics at the University of Washington. Liz Turner is an associate professor of biostatistics and bioinformatics at Duke University.

September 12, 2024: NIH Collaboratory Biostatisticians Evaluate Analytic Models for Individually Randomized Group Treatment Trials

Posted on by Damon Seils

Headshot of Dr. Jonathan Moyer
Dr. Jonathan Moyer

To avoid inflation in the rate of type 1 error, or false positives, in individually randomized group treatment (IRGT) trials, researchers should choose an analytic model that accounts for the correlations in outcome measures that arise when study participants receive an intervention from the same source, according to a report from the NIH Pragmatic Trials Collaboratory’s Biostatistics and Study Design Core.

The report was published online ahead of print in Statistics in Medicine.

Many IRGT trials randomly assign individuals to study arms but deliver the study intervention through shared “agents,” such as clinicians, therapists, or trainers. After randomization, interactions between participants who share the same agent can lead to correlations in study outcomes. The delivery agents may be nested in or crossed with study arm, and participants may interact with a single agent or multiple agents. There has been no systematic effort to identify the appropriate analytic models for these complex study designs.

To address this knowledge gap, members of the NIH Collaboratory’s Biostatistics and Study Design Core conducted a simulation study to examine the performance of a variety of analytic models for IRGT trials in which complex clustering arises from participants interacting with multiple agents or single agents in both nested and crossed designs. They found substantial inflation in the type I error rate in studies with nested designs when the analytic model did not account for participants interacting with multiple agents.

Read the full article.

This article is the latest in a series of reports completed this year by members of the Biostatistics and Study Design Core to explore analytic approaches to clinical trials with complex clustering and other novel design features:

Lead author Jonathan Moyer, a statistician in the NIH Office of Disease Prevention, led a discussion of complex clustering in pragmatic trials in a session of the NIH Collaboratory’s weekly Rethinking Clinical Trials webinar series: “The Perils and Pitfalls of Complex Clustering in Pragmatic Trials.”

Learn more about the NIH Collaboratory’s Biostatistics and Study Design Core.

August 19, 2024: Ethics Core and Biostatistics Core Guide Newest Trials Through Planning Phase

Posted on by Damon Seils

Leaders of 2 of the NIH Pragmatic Trials Collaboratory’s long-standing Core Working Groups recently shared updates from their work with the newest cohort of NIH Collaboratory Trials. We spoke with them during the NIH Collaboratory’s 2024 Annual Steering Committee Meeting in May.

Over the last year, the Ethics and Regulatory Core engaged in a formal onboarding process with the program’s 9 newest pragmatic trials, consulting with the investigators about their trial planning and implementation. Cochairs Stephanie Morain and Pearl O’Rourke summarized several of the ongoing and emerging challenges.

“One of the challenges we’re continuing to see is understanding what are the appropriate duties that institutions and investigators have in the context of a [pragmatic clinical trial],” said Morain. “One concrete area is in data and safety monitoring. What kinds of issues need to be monitored as adverse events? How do we think about them as being related to the trial vs relating to the background care?” she added.

Onboarding documentation from the Ethics and Regulatory Core’s consultations with the NIH Collaboratory Trials is available on our Data and Resource Sharing page.

We also spoke with Liz Turner and Patrick Heagerty, cochairs of the Biostatistics and Study Design Core. They have spent the past year advising the NIH Collaboratory Trial investigators on key study design challenges.

“Many of these studies have individually randomized patients but then they’re studying implementation pathways when they implement through a specific person that puts them in groups—these are individually randomized group treatment trials,” said Heagerty. “Several of the studies didn’t see that, and so we helped them see it and we helped them work through how to adapt their analysis and modify their sample size work to ensure the trial was properly sized,” he explained.

In addition to consultations with the NIH Collaboratory Trials, the Biostatistics and Study Design Core continues to develop and innovate pragmatic trials methodology.

Learn more about the Core Working Groups.

October 4, 2023: Special Biostatistics Grand Rounds Series Begins Friday With Rigorous Methods for Hybrid Studies

Posted on by Damon Seils

In this Friday’s PCT Grand Rounds, David Murray of the NIH Office of Disease Prevention will kick off our special series, Advances in the Design and Analysis of Pragmatic Clinical Trials, with his presentation, “Hybrid Studies Should Not Sacrifice Rigorous Methods.” The session will be held on Friday, October 6, at 1:00 pm eastern and will be moderated by Jonathan Moyer.

Murray is the NIH associate director for prevention and the director of the Office of Disease Prevention. He is a longtime member of the NIH Pragmatic Trials Collaboratory’s Biostatistics and Study Design Core. This session’s moderator, Jon Moyer, is a statistician in the Office of Disease Prevention.

Join the online meeting.

This special Grand Rounds series will include additional moderated webinar discussions that bring together biostatisticians, clinical trials methodologists, and investigators to discuss challenges and share lessons learned in the design, implementation, and analysis of pragmatic trials. Download the series flyer and see the full schedule below.

All sessions are free and open to the public; no registration is required.

September 5, 2023: NIH Pragmatic Trials Collaboratory Announces Grand Rounds Series on Design and Analysis of Pragmatic Clinical Trials

Posted on by Damon Seils

Promotional graphic showing details of the upcoming sessions of special Grand Rounds series, "Advances in the Design and Analysis of Pragmatic Clinical Trials"The NIH Pragmatic Trials Collaboratory is launching a special Grand Rounds series to share advances in the design and analysis of pragmatic clinical trials.

Join us on the first Friday of each month, October through January, to hear the latest best practices and explore emerging questions with experts from the program’s Biostatistics and Study Design Core.

Over the past decade, the Core has worked with investigators to fine-tune study designs, develop rigorous analysis plans, and offer guidance to the broader community of researchers who are planning pragmatic trials. With this new Grand Rounds series, the Core is bringing together biostatisticians, clinical trials methodologists, and investigators to discuss challenges and share lessons learned in the design, implementation, and analysis of pragmatic trials.

The webinar series, Advances in the Design and Analysis of Pragmatic Clinical Trials, will kick off on Friday, October 6, at 1:00 pm ET with a presentation on design and analysis considerations for implementation trials by David Murray, NIH associate director for disease prevention and director of the NIH Office of Disease Prevention.

The series will include 3 additional moderated webinar discussions. These sessions will focus on a range of topics, including complex clustering, best practices in the design and analysis of stepped-wedge trials, and handling missing data in cluster randomized trials.

Download the series flyer and see the full schedule below:

All sessions are free and open to the public; no registration is required. Recordings will be archived on the Rethinking Clinical Trials website.

April 10, 2023: Li Receives New PCORI Award to Develop Causal Inference Methods for Stepped-Wedge Cluster Randomized Trials

Posted on by Damon Seils

Headshot of Dr. Fan Li
Dr. Fan Li

Dr. Fan Li, a member of the NIH Pragmatic Trials Collaboratory’s Biostatistics and Study Design Core since 2013, has received approval of a 3-year funding award from the Patient-Centered Outcomes Research Institute (PCORI) to develop causal inference methods for stepped-wedge cluster randomized trials—a design that has been increasingly adopted in pragmatic trials. Li is an assistant professor of biostatistics at the Yale School of Public Health.

The new study, entitled “Toward Improved Design and Analysis of Stepped Wedge Trials: An Estimand-Aligned and Efficiency-Focused Framework,” will contribute new methods and software for planning and analyzing stepped-wedge cluster randomized trials that enable investigators to (a) target transparent causal estimands under the counterfactual outcomes framework and (b) to leverage baseline information for achieving higher statistical efficiency.

This is Li’s second PCORI award. Read a summary of his previous PCORI award.

An estimand is a precise description of the treatment effect reflecting the scientific question, and is ideally a model-free concept. The research team will contribute weighted average effect estimands to quantify treatment effect evidence by recognizing that unequal cluster sizes may contribute to variations of treatment effects in each cluster-period. In addition, pragmatic trials that adopt a stepped-wedge cluster randomized design frequently collect baseline data on the patient-centered outcomes and/or patient-level characteristics. The research team will study and operationalize estimand-aligned methods that effectively leverage such baseline variables through parametric regression and nonparametric machine learning methods.

Li has assembled a multidisciplinary team for this study, including Dr. Patrick Heagerty, professor of biostatistics at the University of Washington and a cochair of the NIH Collaboratory’s Biostatistics and Study Design Core. In addition, Drs. Jeffrey Jarvik, principal investigator of the NIH Collaboratory’s LIRE NIH Collaboratory Trial, and Douglas Zatzick, principal investigator of the TSOS NIH Collaboratory Trial, serve as stakeholders of the study. The stakeholder team also includes colleagues from the NIA IMPACT Collaboratory, Drs. Thomas Travison and Monica Taljaard.