Adrian F. Hernandez, MD, MHS
Professor of Medicine
Vice Dean for Clinical Research
Duke University School of Medicine
Lesley H. Curtis, PhD
Chair and Professor
Department of Population Health Sciences
Duke University School of Medicine
Interim Executive Director, Duke Clinical Research Institute
Kevin Weinfurt, PhD
Professor and Vice Chair of Research
Department of Population Health Sciences
Duke University School of Medicine
Topic
NIH Collaboratory: Looking Back, Looking Forward
Keywords
Embedded pragmatic clinical trials; ePCTs; NIH Collaboratory; Health care systems research; NIH Collaboratory Trials; Living Textbook; HEAL Initiative; Coordinating Center; Research dissemination; Learning health systems; Real-world evidence
Key Points
The NIH Collaboratory program provides a vantage point for the transformation of embedded clinical research.
How can we harmonize the different ideas about what it is to be “pragmatic” for NIH study sections, IRBs, and DSMB reviews? For example, if your DSMB isn’t knowledgeable about PCTs, you could end up with a very explanatory trial.
A willingness to share imperfections is an important part of learning and helps the clinical trial ecosystem evolve.
An important future topic would be how the NIH Collaboratory and PCORnet fit together.
Michelle N. Meyer, PhD, JD
Assistant Professor & Associate Director, Research Ethics
Center for Translational Bioethics & Health Care Policy
Faculty Co-Director, Behavioral Insights Team
Steele Institute for Health Innovation, Geisinger
Topic
Objecting to Experiments that Compare Two Unobjectionable Policies or Treatments: Implications for Comparative Effectiveness and Other Pragmatic Clinical Trials
Keywords
A vs B trials; Comparative effectiveness research; Clinical equipoise; Randomization; Learning health system
Key Points
Healthcare delivery systems often have an ethical obligation to experiment in order to determine the effects of their policies and treatments on stakeholders. A/B experiments conducted within health systems are intended to increase quality and safety, decrease waste or lower costs, and reduce inequity and injustice.
The “A/B effect” is the approval of untested policies or treatments (A or B) being universally implemented but disapproval of randomized experiments (A/B tests) to determine which of those policies or treatments is superior.
Experimentation aversion may be an important barrier to evidence-based practice.
Discussion Themes
Do you think the objection to random assignment is related to a sense that it is not “random?”
A potential solution to the “A/B effect” is to let patients be partners in improving healthcare by explaining that “we don’t know if A or B is better. Would you be willing to help us find out?”
Read Dr. Meyer and colleagues’ open access article in the journal Proceedings of the National Academy of Sciences (May 2019): Objecting to experiments that compare two unobjectionable policies or treatments.
The GGC4H NIH Collaboratory Trial, now in its implementation phase, has begun enrollment of study participants. Congratulations to Drs. Kuklinski, Sterling, and Catalano and the entire GGC4H study team!
GGC4H is a cluster-randomized trial that is testing the feasibility and effectiveness of implementing Guiding Good Choices—a universal evidence-based anticipatory guidance curriculum for parents of early adolescents—in three large, integrated healthcare systems serving socioeconomically diverse families. In prior community trials, the Guiding Good Choices curriculum has been shown to prevent adolescent substance use, depressive symptoms, and delinquent behavior. This study offers an opportunity to test the intervention’s effectiveness with respect to improving adolescent behavioral health outcomes when implemented at scale in pediatric primary care within a pragmatic trial.
The NIH Health Care Systems Research Collaboratory program is excited to announce that it has received funding to serve as the Resource Coordinating Center for a new group of large-scale embedded pragmatic clinical trials (ePCTs) on pain management and reducing opioid prescribing. As part of the NIH Collaboratory, the Pragmatic and Implementation Studies for the Management of Pain to Reduce Opioid Prescribing (PRISM) Resource Coordinating Center will provide technical support and pragmatic trial expertise for the research that this program funds. PRISM trials will determine the effectiveness of multiple non-opioid interventions for treating pain and assess the impact of implementing interventions or guidelines to improve pain management and reduce reliance on opioids.
The PRISM Resource Coordinating Center funding and new research awards, described below, are part of the NIH’s Helping to End Addiction Long-term Initiative (NIH HEAL Initiative). This federal research initiative, launched in early 2018 by NIH Director Francis S. Collins, aims to apply scientific solutions to improve treatments for chronic pain, curb the rates of opioid use disorder and overdose, and achieve long-term recovery for opioid addiction.
“The NIH Collaboratory Coordinating Center is excited to be supporting these novel pragmatic trials that address an urgent health crisis. We hope the patients, clinicians, researchers, and health systems will benefit from knowledge we’ve gained supporting complex trials embedded in health care systems over the past 7 years, which will help deliver improvements in pain management to the American public faster.” – Adrian Hernandez, MD, MHS, Vice Dean for Clinical Research, Duke University School of Medicine.
The PRISM awards total approximately $35.7 million and are supported by 8 participating NIH institutes, centers, and offices. With these awards, the NIH Collaboratory will add 4 new large-scale ePCTs to its portfolio of innovative NIH Collaboratory Trials. The trials will be conducted at Boston Medical Center, Massachusetts; Kaiser Foundation Research Institute, California; Mayo Clinic, Minnesota; and University of Iowa. They include:
Non-pharmacological Options in postoperative Hospital-based And Rehabilitation pain Management (NOHARM) pragmatic clinical trial
Fibromyalgia TENS in Physical Therapy Study (TIPS): An embedded pragmatic clinical trial
Group-based mindfulness for patients with chronic low back pain in the primary care setting
Pragmatic Trial of Acupuncture for Chronic Low Back Pain in Older Adults
The NIH Collaboratory aims to improve the way clinical trials are conducted by creating a new infrastructure for collaborative research with healthcare systems. The Collaboratory has 5 Core Working Groups of experts that help research teams address challenges of conducting research embedded in clinical care, and they collect and disseminate knowledge and best practices learned throughout the process. The ultimate goal is to ensure that healthcare providers and patients can make decisions based on the best available clinical evidence.
The Coordinating Center of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory is supported by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director. Read more about the NIH Collaboratory Trials and the Core Working Groups.
Rachel L. Winer, PhD
University of Washington
School of Public Health
Department of Epidemiology
Diana S.M. Buist, PhD, MPH
Kaiser Permanente Washington Health Research Institute
Topic
Designing & Testing the Future of Home-based Cervical Cancer Screening: Results from a Collaborative Academic-Embedded Delivery System Pragmatic Randomized Trial
Keywords
Embedded pragmatic clinical trial; Cervical cancer screening; Human papilloma virus
Key Points
The aim of the Home-Based Options to Make Screening Easier (HOME) pragmatic randomized trial was to compare the effectiveness of 2 approaches to increasing cervical cancer screening among women 30-64 years of age who are overdue for cervical cancer screening.
Challenges of embedded pragmatic trials reported by the study team include:
Discussions with lab, primary care, OB/GYN, and prevention and outreach teams
Negotiation about the target population
Aligning with evolving clinical guidelines
Engaging multiple clinical champions
Extensive back and forth with IRB for approval
The study team also conducted semi-structured interviews to understand women’s attitudes, emotional responses, and informational needs after receiving a positive kit result and completing recommended follow up.
Discussion Themes
Were you able to assess the impact of this intervention on the clinic staff?
To help move the field forward, there is a need for more publications and more education of peer reviewers and funders about the challenges of conducting embedded pragmatic trials.
The investigators of HiLo, an NIH Collaboratory Trial, have received approval to move from the planning phase to the implementation phase of their study. Congratulations to Dr. Myles Wolf and the HiLo study team for their excellent work!
HiLo (Pragmatic Trial of Higher vs. Lower Serum Phosphate Targets in Patients Undergoing Hemodialysis) is designed to answer the question of what is the optimal level of serum phosphate for patients with end-stage renal disease (ESRD) who are undergoing hemodialysis. In an effort to improve clinical outcomes, current practice guidelines advocate aggressive treatment of high blood phosphate to near normal levels using dietary phosphate binders and restrictive diets. Yet, the optimal phosphate target remains unknown, and potential harms of aggressive treatment have not been definitively identified. HiLo is the first formal clinical research study to evaluate this important question. The study team is planning the first wave of site activations with the goal of beginning enrollment at 10 dialysis centers in the Raleigh-Durham area in October or November.
We recently asked Dr. Wolf to reflect on the transition of the HiLo trial.
Were there any surprises during the study’s planning phase?
How much work was required to plan a large pragmatic trial! Fortunately, we have a superb team of investigators and study staff who are deeply invested in the trial, deep expertise at the Duke Clinical Research Institute, full engagement of our partners at DaVita and the University of Utah, invaluable insight from our Patient Ambassadors from the American Association of Kidney Patients, and unwavering support from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and the NIH Collaboratory.
What is an example of a challenge that you were able to overcome with the help of a Core group?
The Ethics and Regulatory Core helped us work through unique challenges related to obtaining individual-level informed consent in a cluster-randomized trial. The Biostatistics and Study Design Core and a number of outside statistical consultants helped us identify a novel solution for designing and analyzing a primary outcome of the trial that best aligns with the study’s clinical goal.
“We hope that the experience we gained from HiLo related to application of novel methods for pragmatic trials will stimulate further innovation and enhance the design of future studies in our field, ultimately for the benefit of kidney patients.” – Dr. Myles Wolf, PI of HiLo
What other key challenges have you faced?
We learned from the Ambassadors on our Patients Advisory Group about how important it will be to convince dialysis facility staff and patients that it is justified and important for the study to reevaluate what has been dogma in ESRD treatment: that serum phosphate must be lowered aggressively. We have had to grapple with how to deploy an electronic process to obtain informed consent remotely—a first in U.S. dialysis studies—given that we will not have on-site study coordinators in the participating dialysis facilities. We also had to develop, refine, and defend our use of a newer statistical approach to HiLo’s primary hierarchical composite outcome of all-cause mortality and all-cause hospitalizations. The approach, which is gaining traction in other areas, has not been used in large-scale trials in nephrology. While the process of preparing for this trial was long and required substantial hard work from a large team of investigators and study staff, we hope that the experience we gained from HiLo related to application of novel methods for pragmatic trials will stimulate further innovation and enhance the design of future studies in our field, ultimately for the benefit of kidney patients.
What words of advice do you have for investigators conducting their first embedded PCT?
Get to know the people—patients and professionals—who need to be invested and will be affected by your study and its outcomes. Understand their interests and concerns even if it goes against what you think you know. These early conversations will help identify hurdles at a time when they can be readily addressed and the study enhanced. Be patient and be prepared to work, and work some more. And ask for more money … pragmatic plus more resources is still pragmatic!
Additional details about the study are on the HiLo website.
NIH Collaboratory Trials begin with a 1-year, milestone-driven planning phase. Projects become eligible to move to the implementation phase after an administrative review of progress toward scientific milestones and feasibility requirements. Throughout the process, the project team interacts with the Core Working Groups and investigators from the other NIH Collaboratory Trials.
HiLo is supported within the NIH Collaboratory by a cooperative agreement from the NIDDK and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about HiLo in the Living Textbook, and learn more about the NIH Collaboratory Trials.
The submission deadline has been extended to October 28, 2019, for a special supplement on embedded health services research in Healthcare: The Journal of Delivery Science and Innovation, the partner journal of AcademyHealth. Embedded research is a critical part of the learning health system in mining and analyzing health system data to improve patient care while also providing generalizable findings to transform the health care system at large.
This special supplement is being supported by the Department of Veterans Affairs Health Services Research & Development and will be published in March 2020. It is expected to feature 10-12 peer-reviewed articles. Ultimately, the supplement will be a resource for those aiming to improve the relevance and use of health research to improve patient care.
For details on relevant topics and how to submit your paper online, visit the journal’s special issue page.
Matthew T. Roe, MD, MHS
Senior Investigator, Professor of Medicine
Duke Clinical Research Institute
Topic
Transforming Medical Evidence Generation with Technology-Enabled Trials
Keywords
Mobile clinical trials; Real-world evidence; Real-world data; Study design; Regulatory oversight; Digital health; Mobile health applications; Biosensors; Electronic health records
Key Points
Digital health applications and electronic health records provide tremendous opportunities for improving trial efficiencies, broadening patient participation, and reducing cost.
Novel approaches that can help reduce data collection burden for study sites include importing EHR data directly into the trial database, collecting patient-reported outcomes through web-based portals, and incorporating digital health data from wearables and biosensors.
To realize the potential of new technology, cross-sectional partnerships are needed among research participants, researchers, biopharma device industries, professional medical associations, insurers, FDA, clinicians, health IT, contract research organizations, and health systems.
Discussion Themes
How many potential patients might we lose if having a smart phone is an inclusion criterion for a clinical study?
How can we ensure that the clinical trial infrastructure is inclusive of minority populations, especially those in rural settings?
What is the role of physicians in reaching a large number of participants who are not near an academic research center?
Ultimately, in clinical trials, the data are what matter and what decisions are based on. We need to understand data quality and standards for the data to be accepted.
Vanita R. Aroda, MD
Director of Diabetes Clinical Research
Brigham & Women’s Hospital
Harvard Medical School
Topic
Oh Yes, We Have Tons of Patients Who Can Do This Study!
Keywords
Patient engagement; Patient recruitment and retention; Clinician engagement; Health care systems; Multicenter clinical trials; Electronic health record
Key Points
Research occurs beyond the silo. Effective large-scale multicenter clinical trial recruitment requires an accessible network of potential participants.
Engage colleagues and the healthcare system as part of the collaborative journey across the trial’s lifecycle.
It is highly recommended to do a role-playing exercise with the study team to prevent fumbles when engaging and recruiting study participants.
The science, the protocols, and the data are all important, but it is the essential human element that makes it all happen.
Discussion Themes
Participant retention is really a continuation of good recruitment and engagement.
Make sure your database query makes clinical sense and is the best fit to answer your study question. Don’t spend time on the wrong data.
What other recruitment opportunities or techniques can sites use after they exhaust their patient panel?
At the May 2019 meeting of the NIH Collaboratory Steering Committee, we talked with Judith Carrithers, coleader of the Ethics and Regulatory Core. The task of the Core is to develop a framework for conducting embedded pragmatic clinical trials (ePCTs) in an ethical manner and in compliance with federal and state regulations. Ms. Carrithers joined the Core last year prior to the start of the yearlong planning phase for 6 new UG3 NIH Collaboratory Trials. We asked her to reflect on the Core’s progress and challenges during the past year.
Please tell us about the Core’s recent accomplishments.
The Ethics and Regulatory Core is learning how to frame ethical and regulatory issues around ePCTs while talking with each study team to learn how their trial is going to work, what informed consent considerations they may have, and, for their population, what makes the most sense within the regulatory framework. By the time I joined, the Core had already gone through the first round of UH3 NIH Collaboratory Trials, and I was able to piggyback on the learning from that experience, which informed our interviews and discussions with the new UG3 studies last summer. The regulatory framework we’re working in is a little black, a little white—and a lot of gray. For ePCTs, and clinical trials in general, within that framework there are things it’s clear you can do and cannot do, and a lot of things where you’re using your best judgment in the context of a study.
“The regulatory framework we’re working in is a little black, a little white—and a lot of gray.”
What we see with pragmatic trials across those conducted in the Collaboratory is that many are clearly minimal-risk studies, so there is the possibility of managing informed consent in a different way. A written consent form is generally required under the federal regulations for studies that present more than minimal risk to participants. But if a trial is minimal risk, we can consider a waiver of consent or alteration of the consent process if traditional written consent affects the practicability of the trial. One focus of the Core’s work has been to study when a waiver or alteration of consent is appropriate in the various types of ePCTs. In addition, we explore what other methods could be used to advise patients that they’ve been enrolled in a research study, such as broadcast notification of the research placed in prominent locations, with contact information for questions.
From the inception of the Collaboratory, both the NIH and the Office for Human Research Protections (OHRP) have been involved in helping work through how to manage these issues in a way that respects individuals enrolled in a trial while also making it possible to conduct the trial without a lengthy informed consent process when it is not required under the regulations. We will continue to look at these issues with the new NIH Collaboratory Trials to get a better feel for emerging patterns. The Core has developed several publications addressing ethics and regulatory considerations for ePCTs, and we will continue to contribute to this growing body of knowledge to share with the larger research community.
What challenges lie ahead?
A big challenge is staying aware of how the regulatory framework may change during the course of the trial, and how those changes affect the conduct of a study. For example, the revised Common Rule impacted the way IRBs review research and investigators conduct their research. It’s also important to remember what we’ve learned as a research community—for example, we’re developing better ways of giving notice to patients that they’re enrolled in a trial. And the challenge in part is that studies have used different methods of notification with varying success, and so we need a way to compile that information into an accessible format to help future study teams decide how to apply those learnings to their study.
Our challenge is to build the grammar, the framework, and the thinking process for ethics and regulatory issues in pragmatic trials. Having resources like the Living Textbook available is helpful for researchers, providing insight into how others are framing these issues and conducting their trials.
Any words of advice for new ePCT investigators?
Sort out what part of the trial is research and what part is clinical care. This is essential for study teams to define so that they know what parts of the trial are subject to the federal regulations. It’s important to segment out and treat the clinical part of the study as clinical care. Within the research part, evaluate how the regulations apply. Think carefully about your trial and work through all the pragmatic pieces, for example:
What access to the electronic health record will you need?
How will you recruit participants?
If consent is required, how will you consent participants?
One of the strengths of the Core is that we’re able to work with study teams while they’re still finalizing the design of the trial, and together build on each others’ experiences, focus on specific issues, and in some cases, change their approach in order to make the study work better in the healthcare setting or with potentially large numbers of enrollees. I think the best resource for new investigators is meeting other researchers who have done this work and hear how they addressed and overcame challenges.
The Coordinating Center of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory is supported by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director. Read more about the Ethics and Regulatory Core in the Living Textbook, and learn more about the NIH Collaboratory's other Core Working Groups.
The GGC4H NIH Collaboratory Trial, now in its implementation phase, has begun enrollment of study participants. Congratulations to Drs. Kuklinski, Sterling, and Catalano and the entire GGC4H study team!
The NIH Health Care Systems Research Collaboratory program is excited to announce that it has received funding to serve as the Resource Coordinating Center for a new group of large-scale embedded pragmatic clinical trials (ePCTs) on pain management and reducing opioid prescribing. As part of the NIH Collaboratory, the 
