February 14, 2023: IMPACT Collaboratory to Host Grand Rounds on Treatment Effect Heterogeneity in Cluster Randomized Trials

Headshot of Dr. Fan Li
Dr. Fan Li

Dr. Fan Li, a member of the NIH Pragmatic Trials Collaboratory’s Biostatistics and Study Design Core, will present “Methods for Designing Cluster Randomized Trials to Detect Treatment Effect Heterogeneity” during IMPACT Grand Rounds on Thursday, February 16, at 12:00 pm eastern. IMPACT Grand Rounds is hosted by the NIA IMPACT Collaboratory.

Fan Li, PhD, is an assistant professor in the Department of Biostatistics at Yale School of Public Health, and faculty member in the Center for Methods in Implementation and Prevention Science and the Yale Center for Analytical Sciences. He is the principal investigator of a Patient-Centered Outcomes Research Institute (PCORI)–funded methods award that investigates new study planning methods and software for testing treatment effect heterogeneity in cluster randomized trials.

Zoom Conferencing
Join from PC, Mac, iOS or Android: https://hebrewseniorlife.zoom.us/j/97344810673
Dial-In:  +1 312 626 6799 (US Toll) or  +1 470 250 9358 (US Toll)
Meeting ID:  973 4481 0673

Read more about this IMPACT Grand Rounds session.

August 16, 2022: Biostatistics Core Develops Tools and Strategies for Common Research Challenges

Head shot of Dr. Patrick HeagertyHead shot of Dr. Liz TurnerIn an interview at the NIH Pragmatic Trials Collaboratory’s annual Steering Committee meeting and 10th anniversary celebration, we asked Dr. Liz Turner and Dr. Patrick Heagerty to reflect on the role of the Biostatistics and Study Design Core Working Group in helping the NIH Collaboratory Trial teams design their trials and analyze the data, and to discuss their focus for the Core's future contributions to pragmatic clinical trials.

Based on your experience working with the NIH Collaboratory Trials, what are some of the common challenges of the Core?

Given the pragmatic nature of the NIH Collaboratory Trials, most use a design that involves some kind of clustering of outcomes. This could be a cluster randomized design or an individually randomized group treatment trial. As a consequence, nearly all projects face the challenge of how to account for clustering in both the design and analysis of the trial.

For the NIH Collaboratory Trials that use a cluster randomized design, one of the most common challenges is deciding between a stepped-wedge design and a standard parallel-arm design. The Core’s recommendation is clear: only use a stepped-wedge design if you have to! Likewise, only use a cluster randomized design if you have to and, if possible, use an individually randomized design. Nevertheless, a cluster randomized design is often the design of choice to address a pragmatic research question, and a stepped-wedge cluster randomized design may be the only way to perform a randomized evaluation of an intervention (for example, when all centers wish to receive the intervention in order to agree to participate in the trial).

From an analysis perspective, common challenges involve how to handle missing outcome data and how to handle longitudinal (that is, repeated) measures data. For both design and analysis, as you can imagine, the COVID-19 pandemic has posed huge challenges, including how to handle the disruption of an ongoing stepped-wedge trial (as in the GGC4H NIH Collaboratory Trial). In short, clustering of outcomes is the biggest theme (and challenge) across the NIH Collaboratory Trials.

What strategies have NIH Collaboratory Trials used to overcome these barriers?

A common strategy used by the NIH Collaboratory Trials to overcome these barriers has been to leverage what we call the “Core group process.” This dynamic process is driven by the NIH Collaboratory Trials and supported by the Core, together with NIH Collaboratory leadership. The process is centered around the monthly Core meeting to which all NIH Collaboratory Trial teams are invited and that involves all Core members. These meetings provide dedicated time for each study team to provide project updates and elicit feedback from the Core and the other NIH Collaboratory Trial teams. In particular, all the study teams are invited to present at least once during the UG3 planning phase and on multiple occasions during the UH3 implementation phase. Core members are also available for ad hoc, smaller group meetings, as requested. What this process allows is for the NIH Collaboratory Trials to present challenges and for us to jointly identify solutions.

How are the NIH Collaboratory Trials’ experiences with the Core helping the field of pragmatic research?

Through the challenges and ideas that have been brought to the Core, the NIH Collaboratory Trials have pushed the field of pragmatic research. In particular, through the Core group process, they have pushed the Core to solve methodological challenges and provide tools to tackle the design issues that arise in the changing research landscape.

Thumbnail image of the COVID-19 checklist

A key example of the Core’s methodological work was inspired by the STOP CRC NIH Collaboratory Trial and is related to the design and analysis choices faced in the unique context of embedded pragmatic trials. This example addresses a common challenge in embedded pragmatic trials, namely how to handle varying cluster sizes, something that arises in so many of the NIH Collaboratory Trials. The research, recently published in Contemporary Clinical Trials, highlights that a seemingly natural analysis in this context may produce a biased inference about intervention effectiveness, which is clearly problematic.

The second example is the Core’s recently published Statistical Analysis Plan Checklist for Addressing COVID-19 Impacts. Development of this tool was inspired by the many challenges faced by the NIH Collaboratory Trials as a result of the COVID-19 pandemic, such as delayed recruitment (as in the BackInAction NIH Collaboratory Trial) and adjustments to how interventions were delivered (as in the ACP PEACE NIH Collaboratory Trial).

What do you think the Core can contribute over the next decade?

The Core has a lot to contribute over the next decade. A key goal is to ensure we are building and diversifying the next generation of statisticians who are experts in pragmatic trials and who can engage deeply in the design and analysis of pragmatic trials embedded in healthcare systems.

To achieve this, we need to continue to bring trainees into the Core, as we have done over the past 6 years, through funded graduate research assistant positions. By doing this, we should be able to not only build the next generation of pragmatic trial experts but also build scholarship in pragmatic trial methodology by identifying methodological gaps needed to be filled so the NIH Collaboratory Trials study teams—and pragmatic trialists in the broader research community—have the best methods available to them.

The opportunity to participate in a cross-institution working group such as ours is surprisingly rare. As a consequence, we are in a unique position to not only build the next generation of experts but also to strength our own collective expertise and knowledge by learning from each other’s perspectives.

June 1, 2022: This Week’s PCT Grand Rounds Will Feature MedSafer Deprescribing Study

Head shot of Dr. Emily McDonaldIn this Friday’s PCT Grand Rounds, Dr. Emily McDonald of McGill University will present “Results of the MedSafer Study: Electronic Decision Support for Deprescribing in Hospitalized Older Adults.” The Grand Rounds session will be held on Friday, June 3, 2022, at 1:00 pm eastern.

The MedSafer study was a cluster randomized clinical trial evaluating the effects of an electronic deprescribing decision support tool on adverse drug events after hospital discharge among older adults with polypharmacy.

Join the online meeting.

May 25, 2022: NOHARM Aims to Change How Patients Manage Pain After Surgery

NOHARM, an NIH Pragmatic Trials Collaboratory Trial, is a cluster-randomized, stepped-wedge trial of a bundled intervention that relies heavily on the electronic health record to encourage perioperative patients to consider using nonpharmacologic approaches to manage their postoperative pain. The goal is to diminish patients’ reliance on opioids after surgery and, as a result, reduce rates of inappropriate and prolonged use of opioids.

Dr. Andrea L. Cheville, Co-PI of the Non-pharmacological Options in postoperative Hospital-based And Rehabilitation pain Management (NOHARM) project, discussed the study in an interview during the NIH Pragmatic Trials Collaboratory’s Steering Committee meeting in April.

NOHARM includes 5 steps with 22 clusters randomized to each step. The clusters are defined by surgical type as well as site. The project includes 4 health systems that fall under the Mayo enterprise and share a common electronic health record.

“We have now gone live with 2 of our steps, a total of about 10 clusters, and the surgical types that are included are transplant, cardiac, pulmonary, gynecologic, C-section, orthopedic, and colorectal surgeries,” said Cheville. “The statisticians from the Collaboratory were truly vital allies in parameterizing our choice of procedures and the stepped-wedge so I really cannot call them out enough.”

In May, the project was set to go live with a third step, the automated assignment of a portal-delivered conversation guide to patients.

“The goal of the guide is to educate patients that they are likely to have pain, that it’s important to take seriously and plan for, the benefits and the potential harms of opioid use, and the availability of equally effective, safer nonpharmacologic options that have additional benefits with respect to function and anxiety and sleep quality,” Cheville said.

NOHARM has engaged more than 36,000 patients with a target of roughly 100,000 patients total. Cheville said the project has learned that patients’ receptivity to different pain management options is dynamic over the course of their perioperative journey.

NOHARM, like other trials, has faced challenges related to COVID-19, and the fact that sites may experience surges at different times. A COVID-19 surge at one site can mean that nurses trained in the intervention are deployed elsewhere or that their ability to ingest new information and change practice patterns is stressed and challenged.

“We are trying to find a happy medium between advancing the trial, ensuring high-fidelity delivery, and consistent implementation across all of our clusters while being very respectful to the taxing demands imposed by the COVID-19 epidemic on all of the stakeholder groups,” Cheville said.

Cheville said being part of the NIH Pragmatic Trials Collaboratory has been extremely beneficial for NOHARM, particularly having access to experts who are available to help with methodology, informatics, the electronic health record, parameterization, and statistics.

“If I had to pinpoint one attribute of the Collaboratory it would be the freedom to share our weaknesses, our fears, the things that keep us up at night, with colleagues who are grappling and troubleshooting exactly the same issues in parallel, who truly are walking the walk. Feeling part of a community, that’s been vital for our own psychosocial well-being,” Cheville said.

View the full video here.

March 30, 2022: Two Weights Make a Wrong: New Article From the Biostatistics and Study Design Core

Contemporary Clinical TrialslsIn a new article from the NIH Pragmatic Trials Collaboratory Biostatistics and Study Design Core, the authors share analytic considerations for cluster randomized trials with hierarchical nesting of participants within clusters. The authors illustrate the problem using theoretical derivations, a simulation study, and data from the STOP CRC NIH Collaboratory Trial as an example.

“We conclude that an analysis using both an exchangeable working correlation matrix and weighting by inverse cluster size, which may be considered the natural analytic approach, can lead to incorrect results. That is, two weights make a wrong. The bias is minimal when there is homogeneity of treatment effects according to cluster size but unacceptable when there is heterogeneity of treatment effects according to cluster size. In addition, we show that only an analysis with an independence working correlation matrix and weighting by inverse cluster size always provides valid results for the UATE [unit average treatment effect] estimand.”

Read the full article.

March 8, 2022: New Article Examines Ethical Considerations in Stepped-Wedge Cluster Randomized Trial Designs

Members of the NIH Pragmatic Trials Collaboratory Ethics and Regulatory Core published a new article in Contemporary Clinical Trials exploring the ethical and epistemic advantages and challenges specific to pragmatic stepped-wedge cluster randomized trial designs.

Contemporary Clinical TrialslsThe article titled “Ethical and Epistemic Issues in the Design and Conduct of Pragmatic Stepped-wedge Cluster Randomized Clinical Trials” was published online ahead of print in a special issue on pragmatic and virtual trials.

The authors draw on real-world examples from 5 NIH Collaboratory Trials: ACP PEACE, LIRE, NOHARM, PRIM-ER, and TSOS. The article examines practical, ethical, and epistemic issues faced by these NIH Collaboratory Trials.

In a traditional parallel cluster randomized trial, some clusters receive the intervention while other clusters do not, potentially depriving this control group of a beneficial intervention.  The stepped-wedge cluster randomized trial (SW-CRT) design takes steps toward minimizing this ethical dilemma by rolling out the intervention to all clusters over a predetermined time schedule.  This type of trial may have logistical and statistical advantages for researchers while allowing all study groups the opportunity to benefit from the intervention.

The unique design of SW-CRTs may also lead to regulatory challenges involving informed consent, bias due to contamination of the control clusters, or changing study conditions due to the extended length of time over which an intervention is rolled out.  The decision to conduct a pragmatic SW-CRT requires careful consideration of these challenges weighed against the possible advantages.

The NIH Pragmatic Trials Collaboratory is supported by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director, and through the NIH HEAL Initiative.

January 19, 2022: PCT Grand Rounds Will Feature PRIM-ER Study of Primary Palliative Care in Emergency Medicine

Headshot of Dr. Corita Grudzen
Dr. Corita Grudzen, principal investigator of PRIM-ER

In this Friday’s PCT Grand Rounds, the principal investigator of PRIM-ER, an NIH Collaboratory Trial, will share an update on the implementation of this innovative pragmatic clinical trial.

Principal investigator Dr. Corita Grudzen of the NYU Grossman School of Medicine will present “Primary Palliative Care for Emergency Medicine.” The Grand Rounds session will be held on Friday, January 21, at 1:00 pm eastern.

PRIM-ER, or the Primary Palliative Care for Emergency Medicine study, is a cluster randomized pragmatic trial of a multidisciplinary primary palliative care intervention that consists of education, clinical decision support, and other elements in a diverse mix of emergency departments in the United States. The intervention is intended to improve the delivery of goal-directed emergency care of older adults.

Join the online meeting.

PRIM-ER is supported within the NIH Collaboratory by a cooperative agreement from the National Institute on Aging.

January 18, 2022: Documentation Available From Ethics and Regulatory Consultation With IMPACt-LBP NIH Collaboratory Trial

NIH Pragmatic Trials Collaboratory logo

Meeting minutes and supplementary materials summarizing a recent discussion of ethics and regulatory issues associated with the new IMPACt-LBP NIH Collaboratory Trial are now available. The consultation took place by video conference and included representation from the study’s principal investigators and project manager, members of the NIH Collaboratory’s Ethics and Regulatory Core, NIH staff, and NIH Collaboratory Coordinating Center personnel.

IMPACt-LBP is a 2-arm cluster randomized trial that will evaluate the effect of first-contact patient referral to physical therapists and doctors of chiropractic for patients with low back pain. The study aims to determine if initial contact with these clinicians will improve physical function and decrease pain, among other outcomes, in patients with a primary complaint of low back pain, when compared with usual medical care. Read more about IMPACt-LBP.

Ethics and regulatory documentation for all of the NIH Collaboratory Trials is available on our Data and Resource Sharing page.

December 14, 2021: A Year of New Insights From the NIH Collaboratory

Collage of journal coversNIH Collaboratory researchers in 2021 shared study results, generated new knowledge, and developed innovative research methods in pragmatic clinical trials. Their work included insights from the Coordinating Center and Core Working Groups, analyses from the NIH Collaboratory Distributed Research Network, and results and methodological approaches from the NIH Collaboratory Trials.

So far this year, the NIH Collaboratory has produced 3 dozen articles in the peer-reviewed literature, including the primary results of the PPACT and TSOS trials, the study design of the Nudge and OPTIMUM studies, insights into the COVID-19 pandemic from the EMBED and ACP PEACE studies, and more:

NIH Collaboratory Coordinating Center

NIH Collaboratory Distributed Research Network

ACP PEACE NIH Collaboratory Trial

BackInAction NIH Collaboratory Trial

EMBED NIH Collaboratory Trial

GRACE NIH Collaboratory Trial

HiLo NIH Collaboratory Trial

LIRE NIH Collaboratory Trial

Nudge NIH Collaboratory Trial

OPTIMUM NIH Collaboratory Trial

PPACT NIH Collaboratory Trial

PRIM-ER NIH Collaboratory Trial

PROVEN NIH Collaboratory Trial

SPOT NIH Collaboratory Trial

TSOS NIH Collaboratory Trials

December 1, 2021: PCT Grand Rounds to Highlight Primary Results of PPACT Study

Head shot of Dr. Lynn DeBar
Dr. Lynn DeBar, principal investigator of PPACT

In this Friday’s PCT Grand Rounds, the principal investigator of the PPACT study, an NIH Collaboratory Trial, will announce the study’s primary results.

Principal investigator Dr. Lynn DeBar of the Kaiser Permanente Washington Health Research Institute, will present “Primary Care-Based Behavioral Treatment for Long Term Opioid Users With Chronic Pain: Primary Results and Lessons Learned From the PPACT Pragmatic Trial.” The Grand Rounds session will be held on Friday, December 3, at 1:00 pm eastern.

PPACT, or the Collaborative Care for Chronic Pain in Primary Care study, was a pragmatic, cluster randomized trial that enrolled 850 patients who were receiving long-term opioid therapy for chronic pain. Patients in the intervention group participated in weekly group sessions that taught a variety of pain self-management techniques. The strategy was compared with usual care.

Join the online meeting.

PPACT was supported within the NIH Collaboratory by the NIH Common Fund and by cooperative agreements from the National Institute of Neurological Disorders and Stroke.