November 8, 2019: Lumbar Imaging with Reporting of Epidemiology: Initial Results and Some Lessons Learned (Jeffrey Jarvik, MD, MPH, Patrick Heagerty, PhD)


Jeffrey (Jerry) G. Jarvik MD MPH
Professor, Radiology, Neurological Surgery and Health Services
Adjunct Professor, Pharmacy and Orthopedics & Sports Medicine
University of Washington

Patrick Heagerty, PhD
Professor and Chair
Department of Biostatistics
University of Washington


Lumbar Imaging with Reporting of Epidemiology: Initial Results and Some Lessons Learned


Embedded pragmatic clinical trials; Radiology imaging; LIRE; Stepped-wedge; Cluster randomization; Epidemiology; Back pain

Key Points

  • The LIRE Demonstration Project evaluated whether prevalence benchmark data inserted into lumbar spine imaging reports would reduce overall spine-related healthcare utilization for patients referred from primary care.
  • The inserted intervention text urges caution when interpreting the presence of certain findings that are common in normal, pain-free volunteers.
  • While the study team found no decrease in spine-related healthcare utilization for the overall cohort, there was a small but potentially important effect on reducing opioid prescriptions.

Discussion Themes

A characteristic of stepped-wedge study design is that it yields two comparisons: between-group comparisons (clinic A vs clinic B) and within-group comparisons. But temporal trends can have an impact and must be adjusted for in the analysis.

For what type of intervention would a stepped-wedge design be suitable?

The hope is for a wider dissemination about interventions where radiologic testing is done and incidental findings are common.

Read more about the LIRE Demonstration Project.

#pctGR, #PragmaticTrials, @Collaboratory1

November 1, 2019: NIH Collaboratory: Looking Back, Looking Forward (Adrian Hernandez, MD, MHS, Lesley Curtis, PhD, Kevin Weinfurt, PhD)


Adrian F. Hernandez, MD, MHS
Professor of Medicine
Vice Dean for Clinical Research
Duke University School of Medicine

Lesley H. Curtis, PhD
Chair and Professor
Department of Population Health Sciences
Duke University School of Medicine
Interim Executive Director, Duke Clinical Research Institute

Kevin Weinfurt, PhD
Professor and Vice Chair of Research
Department of Population Health Sciences
Duke University School of Medicine


NIH Collaboratory: Looking Back, Looking Forward


Embedded pragmatic clinical trials; ePCTs; NIH Collaboratory; Health care systems research; Demonstration Projects; Living Textbook; HEAL Initiative; Coordinating Center; Research dissemination; Learning health systems; Real-world evidence

Key Points

Discussion Themes

How can we harmonize the different ideas about what it is to be “pragmatic” for NIH study sections, IRBs, and DSMB reviews? For example, if your DSMB isn’t knowledgeable about PCTs, you could end up with a very explanatory trial.

A willingness to share imperfections is an important part of learning and helps the clinical trial ecosystem evolve.

An important future topic would be how the NIH Collaboratory and PCORnet fit together.

Read more about the NIH Collaboratory Program and the Living Textbook of Pragmatic Clinical Trials.

#pctGR, #PragmaticTrials, @Collaboratory1, @texhern, @lmhcurtis, @KevinWeinfurt

November 1, 2019: Collaboratory PIs Give Advice to New Pragmatic Trials

The NIH Health Care Systems Research Collaboratory program is welcoming new embedded pragmatic clinical trials (ePCTs) on pain management and reducing opioid prescribing as part of the NIH’s Helping to End Addiction Long-term Initiative (NIH HEAL Initiative). At the same time, many of the Collaboratory’s early Demonstration Projects are completed and now publishing results. At a recent Collaboratory Steering Committee meeting, we spoke with four of the principal investigators (PIs) of completed trials to glean advice for new trials.

What words of advice do you have for investigators conducting their first embedded PCT?

Dr. Laura Dember, PI of the Time to Reduce Mortality in End-Stage Renal Disease (TiME) trial (NCT02019225).

Dr. Dember: “An important message is that progress in making big changes to how trials are conducted is going to be incremental. We need to be aspirational, but we also need to recognize that we’re not going to solve all of the problems with one try. We should not be discouraged by the things that don’t work, but rather keep moving forward by building on what we’ve learned.”

Dr. Lynn Debar, PI of the Collaborative Care for Chronic Pain in Primary Care (PPACT) trial (NCT02113592).

Dr. DeBar: “Regarding stakeholder engagement, recognize that these are really complex adaptive systems, and sometimes, more resources and harder pushing is not the way to accomplish your goal. You have to really think through how to communicate: where and how you push requires thinking about all the stakeholders involved.”

Dr. Beverly Green, Co-PI of the Strategies and Opportunities to Stop Colorectal Cancer (STOP CRC) trial (NCT01742065).

Dr. Green: “The collaboration part of bringing the investigators together to figure out the pieces of the puzzle has really been valuable to me. Really try to understand stakeholder engagement, the values of the people involved, and work to align them. This way, when you get it right, embedded pragmatic research can save money, improve health, and be a better experience for staff and patients. It’s a win-win.”

Dr. Susan Huang, PI of the Active Bathing to Eliminate (ABATE) Infection trial (NCT02063867).

Dr. Huang: “Communication is so important because if you have stakeholders at the table, not all of them will have the exact vantage point as you, and you will learn by communicating. And, if you’re learning different things and reaching different forks in the road that require decision-making and troubleshooting, it’s only by talking about them that you get everybody to really understand what the goal is. And in this way, you come up with better solutions than you could come up with yourself.”

What was the biggest lesson learned in conducting your trial?

Dr. Dember:  “It’s important to effectively engage with people on the ground. The extent to which this is possible depends on the setting, but, at least for some interventions, being able to engage with the people who are actually implementing the trial procedures is critical.”

Dr. Huang: “The biggest lesson is the critical nature of the partnership and the investment involved in maintaining it. The strength of our trial, and our ability to conduct it, was dependent on a strong partnership.”

Dr. Debar: “Vertical integration is important: you need high level buy-in, but you also need buy-in from people on the ground who are providing care.”

Dr. Green: “Someone in NIH did the math regarding how much an implementation study costs per participant, and he said, ‘That has to be cheaper.’ This prompted me to think about what things could be answered with embedded research. As a clinician, day to day, I am always making decisions that I don’t really know the answer because there isn’t enough evidence. There are so many questions that need to be answered, and embedded pragmatic research is a great starting point for getting more answers. We’ll never get there by doing this one trial at a time with traditional research.”

Congratulations to all four investigators for their groundbreaking work on their Demonstration Projects!

October 30, 2019: Baseline Covariate Imbalance Influences Treatment Effect Bias in Cluster Randomized Trials

In a study supported by the NIH Collaboratory, researchers found that imbalance in individual-level baseline covariates influences bias in the observed treatment effect in cluster randomized trials. Using race as an example, the study highlights the importance of reducing covariate imbalance in the design stage of cluster randomized trials and of using statistical analysis techniques to minimize the resulting bias.

The innovative study, published in Contemporary Clinical Trials, used computer simulation models validated by real-data simulations from a large clinical trial to examine the influence of baseline covariate imbalance on treatment effect bias. They found that bias was proportional to the degree of baseline covariate imbalance and the covariate effect size. In the simulations, trials with larger numbers of clusters had less covariate imbalance. Statistical models that adjusted for important baseline confounders were more effective than unadjusted models in minimizing bias.

The authors recommend several design approaches and statistical analysis techniques for both reducing covariate imbalance and minimizing bias. Using the results of available prior data can help researchers identify important baseline confounders when designing cluster randomized trials.

This work was supported within the NIH Collaboratory by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director, and by a research supplement from the NIH Common Fund to promote diversity in health-related research.

October 28, 2019: Latest Ethics and Regulatory Updates from Demonstration Projects Available

Six Collaboratory Demonstration Projects—ACP PEACE, EMBED, GGC4H, HiLo, Nudge, and PRIM-ER—have recently transitioned from the planning to implementation phase of their embedded pragmatic clinical trial (ePCT). During the transition, study teams reviewed and updated their ethics and regulatory meeting minutes from discussions with the Ethics and Regulatory Core. The minutes describe ethics and regulatory issues the trials have encountered, along with approaches the trials are using for informed consent, HIPAA, and monitoring and oversight:

Ethics and regulatory issues can pose challenges to embedded pragmatic trials because of the unique nature of clinical research conducted in the setting of routine clinical care. The Ethics and Regulatory Core provides assistance to study teams as they navigate the ethics and regulatory landscape of ePCTs.

October 15, 2019: Postdoctoral Fellowship in the Ethics and Regulatory Aspects of Pragmatic Clinical Trials at Johns Hopkins

The Johns Hopkins Berman Institute of Bioethics invites applications for a Postdoctoral Fellowship in the Ethics and Regulatory Aspects of Pragmatic Clinical Trials. This position includes pursuing independent research, working alongside faculty members involved with the ethics and regulatory aspects of large-scale pragmatic clinical trials (PCTs), and participating in the Hecht-Levi Postdoctoral Fellowship in Bioethics. The postdoctoral fellow is expected to pursue one or more projects addressing the ethics and regulatory aspects of PCTs in collaboration with Berman Institute faculty members. The Fellow will actively engage with the Ethics and Regulatory Core of the NIH Health Care Systems Research Collaboratory and the Pragmatic and Implementation Studies for the Management of Pain to Reduce Opioid Prescribing (PRISM) Resource Coordinating Center.

Qualifications: Applications are welcome from candidates who will have an MD, PhD, or JD or their equivalent by the start date of the fellowship. Applicants should not have completed their terminal degree more than 3 years before the start date of the appointment. Physicians should not have completed a formal residency training program more than 3 years by the start date of the appointment.

Start date: September 1, 2020.

Terms of Appointment: The fellowship is guaranteed for 1 year with the expectation of a second year of funding, contingent on review. Applicants may not be employed by another institution and are expected to be in residence for the duration of the appointment.

How to apply: For details on how to apply see: Applications must be submitted by December 16, 2019.

October 11, 2019: Objecting to Experiments that Compare Two Unobjectionable Policies or Treatments: Implications for Comparative Effectiveness and Other Pragmatic Clinical Trials (Michelle Meyer, PhD, JD)


Michelle N. Meyer, PhD, JD
Assistant Professor & Associate Director, Research Ethics
Center for Translational Bioethics & Health Care Policy
Faculty Co-Director, Behavioral Insights Team
Steele Institute for Health Innovation, Geisinger


Objecting to Experiments that Compare Two Unobjectionable Policies or Treatments: Implications for Comparative Effectiveness and Other Pragmatic Clinical Trials


A vs B trials; Comparative effectiveness research; Clinical equipoise; Randomization; Learning health system

Key Points

  • Healthcare delivery systems often have an ethical obligation to experiment in order to determine the effects of their policies and treatments on stakeholders. A/B experiments conducted within health systems are intended to increase quality and safety, decrease waste or lower costs, and reduce inequity and injustice.
  • The “A/B effect” is the approval of untested policies or treatments (A or B) being universally implemented but disapproval of randomized experiments (A/B tests) to determine which of those policies or treatments is superior.
  • Experimentation aversion may be an important barrier to evidence-based practice.

Discussion Themes

Do you think the objection to random assignment is related to a sense that it is not “random?”

A potential solution to the “A/B effect” is to let patients be partners in improving healthcare by explaining that “we don’t know if A or B is better. Would you be willing to help us find out?”

Read Dr. Meyer and colleagues’ open access article in the journal Proceedings of the National Academy of Sciences (May 2019): Objecting to experiments that compare two unobjectionable policies or treatments.

#pctGR, @Collaboratory1

October 11, 2019: Guiding Good Choices for Health (GGC4H) Pragmatic Trial Enrolls First Participant

The GGC4H Demonstration Project, now in its implementation phase, has begun enrollment of study participants. Congratulations to Drs. Kuklinski, Sterling, and Catalano and the entire GGC4H study team!

GGC4H is a cluster-randomized trial that is testing the feasibility and effectiveness of implementing Guiding Good Choices—a universal evidence-based anticipatory guidance curriculum for parents of early adolescents—in three large, integrated healthcare systems serving socioeconomically diverse families. In prior community trials, the Guiding Good Choices curriculum has been shown to prevent adolescent substance use, depressive symptoms, and delinquent behavior. This study offers an opportunity to test the intervention’s effectiveness with respect to improving adolescent behavioral health outcomes when implemented at scale in pediatric primary care within a pragmatic trial.

Read more about GGC4H:

GGC4H Demonstration Project

PI Interview

PCT Grand Rounds webinar

October 4, 2019: Ascertaining Death and Hospitalization Endpoints: The TRANSFORM-HF Experience (Eric Eisenstein, DBA, Kevin Anstrom, PhD)


Eric L. Eisenstein, DBA
Associate Professor in Medicine
Duke University School of Medicine

Kevin J. Anstrom, PhD
Professor of Biostatistics and Bioinformatics
Director of Biostatistics, Duke Clinical Research Institute
Duke University School of Medicine


Ascertaining Death and Hospitalization Endpoints: The TRANSFORM-HF Experience


Clinical endpoints: Ascertaining death; Hospitalization; TRANSFORM-HF; National Death Index

Key Points

  • When patient deaths occur outside the care setting, the cause of death may not be reliably documented. For researchers, the challenges of measuring deaths include the lack of a national death data source and incomplete or hard-to-access sources.
  • The death identification and adjudication process differs for explanatory versus pragmatic trials, and has implications for how death endpoints are acquired and measured.
  • The TRANSFORM-HF pragmatic trial is comparing the effects of treatment strategies on long-term outcomes for hospitalized patients with heart failure. The primary study endpoint is all-cause mortality, which is ascertained and verified using a hybrid approach at the clinical site and call center, and includes searching the National Death Index data.

Discussion Themes

What are the tradeoffs in making endpoint ascertainment more simple?

If using a hybrid death data collection strategy, how are discrepancies adjudicated?

Use of call centers that coordinate follow-up patient contact and data collection is a valid approach that ensures a single point of contact for patients or proxies and care providers. This approach should also be supplemented with redundant data sources.

Read more in the Living Textbook about Using Death as an Endpoint and Inpatient Endpoints in Pragmatic Clinical Trials.

#pctGR, @Collaboratory1, @DCRINews

September 27, 2019: NIH Collaboratory to Serve as Coordinating Center for New Pragmatic Trials Addressing Opioid Crisis

The NIH Health Care Systems Research Collaboratory program is excited to announce that it has received funding to serve as the Resource Coordinating Center for a new group of large-scale embedded pragmatic clinical trials (ePCTs) on pain management and reducing opioid prescribing. As part of the NIH Collaboratory, the Pragmatic and Implementation Studies for the Management of Pain to Reduce Opioid Prescribing (PRISM) Resource Coordinating Center will provide technical support and pragmatic trial expertise for the research that this program funds. PRISM trials will determine the effectiveness of multiple non-opioid interventions for treating pain and assess the impact of implementing interventions or guidelines to improve pain management and reduce reliance on opioids.

The PRISM Resource Coordinating Center funding and new research awards, described below, are part of the NIH’s Helping to End Addiction Long-term Initiative (NIH HEAL Initiative). This federal research initiative, launched in early 2018 by NIH Director Francis S. Collins, aims to apply scientific solutions to improve treatments for chronic pain, curb the rates of opioid use disorder and overdose, and achieve long-term recovery for opioid addiction.

“The NIH Collaboratory Coordinating Center is excited to be supporting these novel pragmatic trials that address an urgent health crisis. We hope the patients, clinicians, researchers, and health systems will benefit from knowledge we’ve gained supporting complex trials embedded in health care systems over the past 7 years, which will help deliver improvements in pain management to the American public faster.” – Adrian Hernandez, MD, MHS, Vice Dean for Clinical Research, Duke University School of Medicine.

The PRISM awards total approximately $35.7 million and are supported by 8 participating NIH institutes, centers, and offices. With these awards, the NIH Collaboratory will add 4 new large-scale ePCTs to its portfolio of innovative Demonstration Projects. The trials will be conducted at Boston Medical Center, Massachusetts; Kaiser Foundation Research Institute, California; Mayo Clinic, Minnesota; and University of Iowa. They include:

  • Non-pharmacological Options in postoperative Hospital-based And Rehabilitation pain Management (NOHARM) pragmatic clinical trial
  • Fibromyalgia TENS in Physical Therapy Study (TIPS): An embedded pragmatic clinical trial
  • Group-based mindfulness for patients with chronic low back pain in the primary care setting
  • Pragmatic Trial of Acupuncture for Chronic Low Back Pain in Older Adults

The NIH Collaboratory aims to improve the way clinical trials are conducted by creating a new infrastructure for collaborative research with healthcare systems. The Collaboratory has 5 Core Working Groups of experts that help research teams address challenges of conducting research embedded in clinical care, and they collect and disseminate knowledge and best practices learned throughout the process. The ultimate goal is to ensure that healthcare providers and patients can make decisions based on the best available clinical evidence.

Related links:

NIH press release

Duke press release

NIH HEAL Initiative website

The Coordinating Center of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory is supported by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director. Read more about the Collaboratory’s Demonstration Projects and the Core Working Groups.