November 1, 2019: Collaboratory PIs Give Advice to New Pragmatic Trials

The NIH Health Care Systems Research Collaboratory program is welcoming new embedded pragmatic clinical trials (ePCTs) on pain management and reducing opioid prescribing as part of the NIH’s Helping to End Addiction Long-term Initiative (NIH HEAL Initiative). At the same time, many of the Collaboratory’s early Demonstration Projects are completed and now publishing results. At a recent Collaboratory Steering Committee meeting, we spoke with four of the principal investigators (PIs) of completed trials to glean advice for new trials.

What words of advice do you have for investigators conducting their first embedded PCT?

Dr. Laura Dember, PI of the Time to Reduce Mortality in End-Stage Renal Disease (TiME) trial (NCT02019225).

Dr. Dember: “An important message is that progress in making big changes to how trials are conducted is going to be incremental. We need to be aspirational, but we also need to recognize that we’re not going to solve all of the problems with one try. We should not be discouraged by the things that don’t work, but rather keep moving forward by building on what we’ve learned.”

Dr. Lynn Debar, PI of the Collaborative Care for Chronic Pain in Primary Care (PPACT) trial (NCT02113592).

Dr. DeBar: “Regarding stakeholder engagement, recognize that these are really complex adaptive systems, and sometimes, more resources and harder pushing is not the way to accomplish your goal. You have to really think through how to communicate: where and how you push requires thinking about all the stakeholders involved.”

Dr. Beverly Green, Co-PI of the Strategies and Opportunities to Stop Colorectal Cancer (STOP CRC) trial (NCT01742065).

Dr. Green: “The collaboration part of bringing the investigators together to figure out the pieces of the puzzle has really been valuable to me. Really try to understand stakeholder engagement, the values of the people involved, and work to align them. This way, when you get it right, embedded pragmatic research can save money, improve health, and be a better experience for staff and patients. It’s a win-win.”

Dr. Susan Huang, PI of the Active Bathing to Eliminate (ABATE) Infection trial (NCT02063867).

Dr. Huang: “Communication is so important because if you have stakeholders at the table, not all of them will have the exact vantage point as you, and you will learn by communicating. And, if you’re learning different things and reaching different forks in the road that require decision-making and troubleshooting, it’s only by talking about them that you get everybody to really understand what the goal is. And in this way, you come up with better solutions than you could come up with yourself.”

What was the biggest lesson learned in conducting your trial?

Dr. Dember:  “It’s important to effectively engage with people on the ground. The extent to which this is possible depends on the setting, but, at least for some interventions, being able to engage with the people who are actually implementing the trial procedures is critical.”

Dr. Huang: “The biggest lesson is the critical nature of the partnership and the investment involved in maintaining it. The strength of our trial, and our ability to conduct it, was dependent on a strong partnership.”

Dr. Debar: “Vertical integration is important: you need high level buy-in, but you also need buy-in from people on the ground who are providing care.”

Dr. Green: “Someone in NIH did the math regarding how much an implementation study costs per participant, and he said, ‘That has to be cheaper.’ This prompted me to think about what things could be answered with embedded research. As a clinician, day to day, I am always making decisions that I don’t really know the answer because there isn’t enough evidence. There are so many questions that need to be answered, and embedded pragmatic research is a great starting point for getting more answers. We’ll never get there by doing this one trial at a time with traditional research.”

Congratulations to all four investigators for their groundbreaking work on their Demonstration Projects!

October 30, 2019: Baseline Covariate Imbalance Influences Treatment Effect Bias in Cluster Randomized Trials

In a study supported by the NIH Collaboratory, researchers found that imbalance in individual-level baseline covariates influences bias in the observed treatment effect in cluster randomized trials. Using race as an example, the study highlights the importance of reducing covariate imbalance in the design stage of cluster randomized trials and of using statistical analysis techniques to minimize the resulting bias.

The innovative study, published in Contemporary Clinical Trials, used computer simulation models validated by real-data simulations from a large clinical trial to examine the influence of baseline covariate imbalance on treatment effect bias. They found that bias was proportional to the degree of baseline covariate imbalance and the covariate effect size. In the simulations, trials with larger numbers of clusters had less covariate imbalance. Statistical models that adjusted for important baseline confounders were more effective than unadjusted models in minimizing bias.

The authors recommend several design approaches and statistical analysis techniques for both reducing covariate imbalance and minimizing bias. Using the results of available prior data can help researchers identify important baseline confounders when designing cluster randomized trials.

This work was supported within the NIH Collaboratory by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director, and by a research supplement from the NIH Common Fund to promote diversity in health-related research.

October 28, 2019: Latest Ethics and Regulatory Updates from Demonstration Projects Available

Six Collaboratory Demonstration Projects—ACP PEACE, EMBED, GGC4H, HiLo, Nudge, and PRIM-ER—have recently transitioned from the planning to implementation phase of their embedded pragmatic clinical trial (ePCT). During the transition, study teams reviewed and updated their ethics and regulatory meeting minutes from discussions with the Ethics and Regulatory Core. The minutes describe ethics and regulatory issues the trials have encountered, along with approaches the trials are using for informed consent, HIPAA, and monitoring and oversight:

Ethics and regulatory issues can pose challenges to embedded pragmatic trials because of the unique nature of clinical research conducted in the setting of routine clinical care. The Ethics and Regulatory Core provides assistance to study teams as they navigate the ethics and regulatory landscape of ePCTs.

October 15, 2019: Postdoctoral Fellowship in the Ethics and Regulatory Aspects of Pragmatic Clinical Trials at Johns Hopkins

The Johns Hopkins Berman Institute of Bioethics invites applications for a Postdoctoral Fellowship in the Ethics and Regulatory Aspects of Pragmatic Clinical Trials. This position includes pursuing independent research, working alongside faculty members involved with the ethics and regulatory aspects of large-scale pragmatic clinical trials (PCTs), and participating in the Hecht-Levi Postdoctoral Fellowship in Bioethics. The postdoctoral fellow is expected to pursue one or more projects addressing the ethics and regulatory aspects of PCTs in collaboration with Berman Institute faculty members. The Fellow will actively engage with the Ethics and Regulatory Core of the NIH Health Care Systems Research Collaboratory and the Pragmatic and Implementation Studies for the Management of Pain to Reduce Opioid Prescribing (PRISM) Resource Coordinating Center.

Qualifications: Applications are welcome from candidates who will have an MD, PhD, or JD or their equivalent by the start date of the fellowship. Applicants should not have completed their terminal degree more than 3 years before the start date of the appointment. Physicians should not have completed a formal residency training program more than 3 years by the start date of the appointment.

Start date: September 1, 2020.

Terms of Appointment: The fellowship is guaranteed for 1 year with the expectation of a second year of funding, contingent on review. Applicants may not be employed by another institution and are expected to be in residence for the duration of the appointment.

How to apply: For details on how to apply see: Applications must be submitted by December 16, 2019.

October 11, 2019: Guiding Good Choices for Health (GGC4H) Pragmatic Trial Enrolls First Participant

The GGC4H Demonstration Project, now in its implementation phase, has begun enrollment of study participants. Congratulations to Drs. Kuklinski, Sterling, and Catalano and the entire GGC4H study team!

GGC4H is a cluster-randomized trial that is testing the feasibility and effectiveness of implementing Guiding Good Choices—a universal evidence-based anticipatory guidance curriculum for parents of early adolescents—in three large, integrated healthcare systems serving socioeconomically diverse families. In prior community trials, the Guiding Good Choices curriculum has been shown to prevent adolescent substance use, depressive symptoms, and delinquent behavior. This study offers an opportunity to test the intervention’s effectiveness with respect to improving adolescent behavioral health outcomes when implemented at scale in pediatric primary care within a pragmatic trial.

Read more about GGC4H:

GGC4H Demonstration Project

PI Interview

PCT Grand Rounds webinar

Podcast October 1, 2019: Preparing for Clinical Trial Data Sharing and Re-use: The New Reality for Researchers (Rebecca Li, PhD, Frank Rockhold, PhD)

In this episode of the NIH Collaboratory Grand Rounds podcast, Dr. Adrian Hernandez sits down with Drs. Rebecca Li and Frank Rockhold to discuss clinical trial data sharing and re-use. In the discussion, Li and Rockhold highlight benefits, potential fears, and the future of data sharing and open science as well as Vivli, a global clinical research data sharing platform.

Click on the recording below to listen to the podcast.

Want to hear more? View the full Grand Rounds presentation.

For alerts about new episodes, subscribe free on iTunes or SoundCloud.

Read the transcript.

Rate this podcast

September 27, 2019: NIH Collaboratory to Serve as Coordinating Center for New Pragmatic Trials Addressing Opioid Crisis

The NIH Health Care Systems Research Collaboratory program is excited to announce that it has received funding to serve as the Resource Coordinating Center for a new group of large-scale embedded pragmatic clinical trials (ePCTs) on pain management and reducing opioid prescribing. As part of the NIH Collaboratory, the Pragmatic and Implementation Studies for the Management of Pain to Reduce Opioid Prescribing (PRISM) Resource Coordinating Center will provide technical support and pragmatic trial expertise for the research that this program funds. PRISM trials will determine the effectiveness of multiple non-opioid interventions for treating pain and assess the impact of implementing interventions or guidelines to improve pain management and reduce reliance on opioids.

The PRISM Resource Coordinating Center funding and new research awards, described below, are part of the NIH’s Helping to End Addiction Long-term Initiative (NIH HEAL Initiative). This federal research initiative, launched in early 2018 by NIH Director Francis S. Collins, aims to apply scientific solutions to improve treatments for chronic pain, curb the rates of opioid use disorder and overdose, and achieve long-term recovery for opioid addiction.

“The NIH Collaboratory Coordinating Center is excited to be supporting these novel pragmatic trials that address an urgent health crisis. We hope the patients, clinicians, researchers, and health systems will benefit from knowledge we’ve gained supporting complex trials embedded in health care systems over the past 7 years, which will help deliver improvements in pain management to the American public faster.” – Adrian Hernandez, MD, MHS, Vice Dean for Clinical Research, Duke University School of Medicine.

The PRISM awards total approximately $35.7 million and are supported by 8 participating NIH institutes, centers, and offices. With these awards, the NIH Collaboratory will add 4 new large-scale ePCTs to its portfolio of innovative Demonstration Projects. The trials will be conducted at Boston Medical Center, Massachusetts; Kaiser Foundation Research Institute, California; Mayo Clinic, Minnesota; and University of Iowa. They include:

  • Non-pharmacological Options in postoperative Hospital-based And Rehabilitation pain Management (NOHARM) pragmatic clinical trial
  • Fibromyalgia TENS in Physical Therapy Study (TIPS): An embedded pragmatic clinical trial
  • Group-based mindfulness for patients with chronic low back pain in the primary care setting
  • Pragmatic Trial of Acupuncture for Chronic Low Back Pain in Older Adults

The NIH Collaboratory aims to improve the way clinical trials are conducted by creating a new infrastructure for collaborative research with healthcare systems. The Collaboratory has 5 Core Working Groups of experts that help research teams address challenges of conducting research embedded in clinical care, and they collect and disseminate knowledge and best practices learned throughout the process. The ultimate goal is to ensure that healthcare providers and patients can make decisions based on the best available clinical evidence.

Related links:

NIH press release

Duke press release

NIH HEAL Initiative website

The Coordinating Center of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory is supported by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director. Read more about the Collaboratory’s Demonstration Projects and the Core Working Groups.

September 23, 2019: NIH Collaboratory Seeks Feedback on the Living Textbook

The NIH Collaboratory Coordinating Center launched an online survey to collect feedback from clinical investigators, healthcare system partners, and the broader research community about the Living Textbook of Pragmatic Clinical Trials. Responses to the survey will help the Coordinating Center enhance its resources for users interested in pragmatic research.

“We are eager to receive feedback on our signature resource, the Living Textbook,” said Dr. Kevin Weinfurt, co–principal investigator of the NIH Collaboratory Coordinating Center. “This feedback will help us achieve our goal of better enabling researchers to build partnerships with healthcare systems and transform how clinical research is conducted.”

The purpose of the Living Textbook, the NIH Collaboratory’s publicly available online resource at, is to share expert consensus regarding standard approaches, best practices, and special considerations in the design, conduct, and reporting of pragmatic clinical trials. The Living Textbook also provides information about the NIH Collaboratory program, its Demonstration Projects, and related initiatives.

The online survey will remain open until October 31, 2019.

September 13, 2019: HiLo Awarded Continuation From Planning to Implementation Phase

The investigators of HiLo, an NIH Collaboratory Demonstration Project, have received approval to move from the planning phase to the implementation phase of their study. Congratulations to Dr. Myles Wolf and the HiLo study team for their excellent work!

HiLo (Pragmatic Trial of Higher vs. Lower Serum Phosphate Targets in Patients Undergoing Hemodialysis) is designed to answer the question of what is the optimal level of serum phosphate for patients with end-stage renal disease (ESRD) who are undergoing hemodialysis. In an effort to improve clinical outcomes, current practice guidelines advocate aggressive treatment of high blood phosphate to near normal levels using dietary phosphate binders and restrictive diets. Yet, the optimal phosphate target remains unknown, and potential harms of aggressive treatment have not been definitively identified. HiLo is the first formal clinical research study to evaluate this important question. The study team is planning the first wave of site activations with the goal of beginning enrollment at 10 dialysis centers in the Raleigh-Durham area in October or November.

We recently asked Dr. Wolf to reflect on the transition of the HiLo trial.

Were there any surprises during the study’s planning phase?

How much work was required to plan a large pragmatic trial! Fortunately, we have a superb team of investigators and study staff who are deeply invested in the trial, deep expertise at the Duke Clinical Research Institute, full engagement of our partners at DaVita and the University of Utah, invaluable insight from our Patient Ambassadors from the American Association of Kidney Patients, and unwavering support from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and the NIH Collaboratory.

What is an example of a challenge that you were able to overcome with the help of a Core group?

The Ethics and Regulatory Core helped us work through unique challenges related to obtaining individual-level informed consent in a cluster-randomized trial. The Biostatistics and Study Design Core and a number of outside statistical consultants helped us identify a novel solution for designing and analyzing a primary outcome of the trial that best aligns with the study’s clinical goal.

“We hope that the experience we gained from HiLo related to application of novel methods for pragmatic trials will stimulate further innovation and enhance the design of future studies in our field, ultimately for the benefit of kidney patients.” – Dr. Myles Wolf, PI of HiLo

What other key challenges have you faced?

We learned from the Ambassadors on our Patients Advisory Group about how important it will be to convince dialysis facility staff and patients that it is justified and important for the study to reevaluate what has been dogma in ESRD treatment: that serum phosphate must be lowered aggressively. We have had to grapple with how to deploy an electronic process to obtain informed consent remotely—a first in U.S. dialysis studies—given that we will not have on-site study coordinators in the participating dialysis facilities. We also had to develop, refine, and defend our use of a newer statistical approach to HiLo’s primary hierarchical composite outcome of all-cause mortality and all-cause hospitalizations. The approach, which is gaining traction in other areas, has not been used in large-scale trials in nephrology. While the process of preparing for this trial was long and required substantial hard work from a large team of investigators and study staff, we hope that the experience we gained from HiLo related to application of novel methods for pragmatic trials will stimulate further innovation and enhance the design of future studies in our field, ultimately for the benefit of kidney patients.

What words of advice do you have for investigators conducting their first embedded PCT?

Get to know the people—patients and professionals—who need to be invested and will be affected by your study and its outcomes. Understand their interests and concerns even if it goes against what you think you know. These early conversations will help identify hurdles at a time when they can be readily addressed and the study enhanced. Be patient and be prepared to work, and work some more. And ask for more money … pragmatic plus more resources is still pragmatic!

Additional details about the study are on the HiLo website.

NIH Collaboratory Demonstration Projects begin with a 1-year, milestone-driven planning phase. Projects become eligible to move to the implementation phase after an administrative review of progress toward scientific milestones and feasibility requirements. Throughout the process, the project team interacts with the Core Working Groups and investigators from the other Demonstration Projects.

HiLo is supported within the NIH Collaboratory by a cooperative agreement from the NIDDK and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about HiLo in the Living Textbook, and learn more about the NIH Collaboratory Demonstration Projects.

September 11, 2019: Deadline Extended for Special Supplement Seeking Papers on Embedded Research


The submission deadline has been extended to October 28, 2019, for a special supplement on embedded health services research in Healthcare: The Journal of Delivery Science and Innovation, the partner journal of AcademyHealth. Embedded research is a critical part of the learning health system in mining and analyzing health system data to improve patient care while also providing generalizable findings to transform the health care system at large.

This special supplement is being supported by the Department of Veterans Affairs Health Services Research & Development and will be published in March 2020. It is expected to feature 10-12 peer-reviewed articles. Ultimately, the supplement will be a resource for those aiming to improve the relevance and use of health research to improve patient care.

For details on relevant topics and how to submit your paper online, visit the journal’s special issue page.