Matthew T. Roe, MD, MHS
Senior Investigator, Professor of Medicine
Duke Clinical Research Institute
Topic
Transforming Medical Evidence Generation with Technology-Enabled Trials
Keywords
Mobile clinical trials; Real-world evidence; Real-world data; Study design; Regulatory oversight; Digital health; Mobile health applications; Biosensors; Electronic health records
Key Points
Digital health applications and electronic health records provide tremendous opportunities for improving trial efficiencies, broadening patient participation, and reducing cost.
Novel approaches that can help reduce data collection burden for study sites include importing EHR data directly into the trial database, collecting patient-reported outcomes through web-based portals, and incorporating digital health data from wearables and biosensors.
To realize the potential of new technology, cross-sectional partnerships are needed among research participants, researchers, biopharma device industries, professional medical associations, insurers, FDA, clinicians, health IT, contract research organizations, and health systems.
Discussion Themes
How many potential patients might we lose if having a smart phone is an inclusion criterion for a clinical study?
How can we ensure that the clinical trial infrastructure is inclusive of minority populations, especially those in rural settings?
What is the role of physicians in reaching a large number of participants who are not near an academic research center?
Ultimately, in clinical trials, the data are what matter and what decisions are based on. We need to understand data quality and standards for the data to be accepted.
Vanita R. Aroda, MD
Director of Diabetes Clinical Research
Brigham & Women’s Hospital
Harvard Medical School
Topic
Oh Yes, We Have Tons of Patients Who Can Do This Study!
Keywords
Patient engagement; Patient recruitment and retention; Clinician engagement; Health care systems; Multicenter clinical trials; Electronic health record
Key Points
Research occurs beyond the silo. Effective large-scale multicenter clinical trial recruitment requires an accessible network of potential participants.
Engage colleagues and the healthcare system as part of the collaborative journey across the trial’s lifecycle.
It is highly recommended to do a role-playing exercise with the study team to prevent fumbles when engaging and recruiting study participants.
The science, the protocols, and the data are all important, but it is the essential human element that makes it all happen.
Discussion Themes
Participant retention is really a continuation of good recruitment and engagement.
Make sure your database query makes clinical sense and is the best fit to answer your study question. Don’t spend time on the wrong data.
What other recruitment opportunities or techniques can sites use after they exhaust their patient panel?
At the May 2019 meeting of the NIH Collaboratory Steering Committee, we talked with Judith Carrithers, coleader of the Ethics and Regulatory Core. The task of the Core is to develop a framework for conducting embedded pragmatic clinical trials (ePCTs) in an ethical manner and in compliance with federal and state regulations. Ms. Carrithers joined the Core last year prior to the start of the yearlong planning phase for 6 new UG3 NIH Collaboratory Trials. We asked her to reflect on the Core’s progress and challenges during the past year.
Please tell us about the Core’s recent accomplishments.
The Ethics and Regulatory Core is learning how to frame ethical and regulatory issues around ePCTs while talking with each study team to learn how their trial is going to work, what informed consent considerations they may have, and, for their population, what makes the most sense within the regulatory framework. By the time I joined, the Core had already gone through the first round of UH3 NIH Collaboratory Trials, and I was able to piggyback on the learning from that experience, which informed our interviews and discussions with the new UG3 studies last summer. The regulatory framework we’re working in is a little black, a little white—and a lot of gray. For ePCTs, and clinical trials in general, within that framework there are things it’s clear you can do and cannot do, and a lot of things where you’re using your best judgment in the context of a study.
“The regulatory framework we’re working in is a little black, a little white—and a lot of gray.”
What we see with pragmatic trials across those conducted in the Collaboratory is that many are clearly minimal-risk studies, so there is the possibility of managing informed consent in a different way. A written consent form is generally required under the federal regulations for studies that present more than minimal risk to participants. But if a trial is minimal risk, we can consider a waiver of consent or alteration of the consent process if traditional written consent affects the practicability of the trial. One focus of the Core’s work has been to study when a waiver or alteration of consent is appropriate in the various types of ePCTs. In addition, we explore what other methods could be used to advise patients that they’ve been enrolled in a research study, such as broadcast notification of the research placed in prominent locations, with contact information for questions.
From the inception of the Collaboratory, both the NIH and the Office for Human Research Protections (OHRP) have been involved in helping work through how to manage these issues in a way that respects individuals enrolled in a trial while also making it possible to conduct the trial without a lengthy informed consent process when it is not required under the regulations. We will continue to look at these issues with the new NIH Collaboratory Trials to get a better feel for emerging patterns. The Core has developed several publications addressing ethics and regulatory considerations for ePCTs, and we will continue to contribute to this growing body of knowledge to share with the larger research community.
What challenges lie ahead?
A big challenge is staying aware of how the regulatory framework may change during the course of the trial, and how those changes affect the conduct of a study. For example, the revised Common Rule impacted the way IRBs review research and investigators conduct their research. It’s also important to remember what we’ve learned as a research community—for example, we’re developing better ways of giving notice to patients that they’re enrolled in a trial. And the challenge in part is that studies have used different methods of notification with varying success, and so we need a way to compile that information into an accessible format to help future study teams decide how to apply those learnings to their study.
Our challenge is to build the grammar, the framework, and the thinking process for ethics and regulatory issues in pragmatic trials. Having resources like the Living Textbook available is helpful for researchers, providing insight into how others are framing these issues and conducting their trials.
Any words of advice for new ePCT investigators?
Sort out what part of the trial is research and what part is clinical care. This is essential for study teams to define so that they know what parts of the trial are subject to the federal regulations. It’s important to segment out and treat the clinical part of the study as clinical care. Within the research part, evaluate how the regulations apply. Think carefully about your trial and work through all the pragmatic pieces, for example:
What access to the electronic health record will you need?
How will you recruit participants?
If consent is required, how will you consent participants?
One of the strengths of the Core is that we’re able to work with study teams while they’re still finalizing the design of the trial, and together build on each others’ experiences, focus on specific issues, and in some cases, change their approach in order to make the study work better in the healthcare setting or with potentially large numbers of enrollees. I think the best resource for new investigators is meeting other researchers who have done this work and hear how they addressed and overcame challenges.
The Coordinating Center of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory is supported by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director. Read more about the Ethics and Regulatory Core in the Living Textbook, and learn more about the NIH Collaboratory's other Core Working Groups.
A new section of the NIH Collaboratory’s Living Textbook of Pragmatic Clinical Trials discusses challenges associated with missing data that result from noncompliance, crossover, and dropout.
Many randomized controlled trials use an intention-to-treat (ITT) analysis to measure the real-world effects of the intervention. The newly published section, Missing Data and Intention-to-Treat Analyses, considers the population-level causal effects in these trials when there is noncompliance or missing outcome data.
“One rationale for the ITT approach is that it evaluates the real-world effects of the intervention. However, a common misconception is that the ITT analysis will be unbiased regardless of crossover or missing data.”
The new section also introduces a white paper from the NIH Collaboratory’s Biostatistics and Study Design Core, “Analyses of Randomized Controlled Trials in the Presence of Noncompliance and Study Dropout.” This working document offers analysts a more detailed discussion of treatment effects in ITT analyses, including a case example and recommended strategies for estimating and reporting both ITT effects and average causal effects.
Murali Doraiswamy, MBBS
Professor of Psychiatry and Behavioral Sciences
Duke School of Medicine
Topic
AI and the Future of Psychiatry
Keywords
Artificial intelligence; Machine learning; Psychiatry; Ethical adoption of technologies; Mental health; Wearables; Mobile health
Key Points
There is growing evidence from randomized controlled trials of the efficacy of using digital tools in mental health diagnosis and treatment.
Could artificial intelligence (AI) and machine learning technologies be used to:
Reduce the stigma associated with mental health treatment?
Predict the risk for future suicide?
Detect Alzheimer’s years before diagnosis?
Categories of AI applications include low-risk apps that measure but do not diagnose, and apps used in diagnosis or treatment that must meet the same high standards of evidence as medications.
Clinicians still struggle with how to integrate patient data from wearable devices. AI technology might help if it could be used to synthesize the data into a risk profile for an individual.
Discussion Themes
What are the roles of stress, exercise, and sleep in mental health, and can autonomic data from wearables help explain the variance in mental health symptoms?
To develop evidence thresholds for AI, we need larger scale public-private partnerships as well as pragmatic trials addressing key clinical questions.
At the May 2019 meeting of the NIH Collaboratory Steering Committee, we talked with Drs. Ted Melnick and Gail D’Onofrio of EMBED, an NIH Collaboratory Trial, to hear about progress and challenges during the UG3 planning phase. The goal of EMBED is to test whether implementation of a user-centered clinical decision support system increases adoption of initiation of buprenorphine/naloxone into the routine emergency care of patients with opioid use disorder. In the UG3 phase, the study team put in place the infrastructure of a pragmatic, multicenter, parallel, group-randomized health IT intervention. EMBED recently transitioned to the UH3 implementation phase and plans to launch the intervention at 20 sites across 5 healthcare systems in August 2019.
“With EMBED, we’re trying to take evidence-based research and implement it to improve practice. EMBED is both a research and patient care project.”
Were there any surprises during the study’s planning phase?
The first surprise came at last year’s Steering Committee meeting, when we met with the Biostatistics and Study Design Core. They encouraged us to change our original study design from stepped-wedge to group-randomized, which we did. We think this advice led to a stronger study. The main reason for this is the group-randomized design’s ability to better account for temporal changes. Since our intervention is being conducted in the middle of an opioid crisis, there are potentially other concurrent interventions that could make it difficult to determine the effect of our intervention. The group-randomized design should give us better insight into whether our intervention is driving behavior change in treating patients with opioid use disorder.
What is an example of a challenge that you were able to overcome with the help of a Core Working Group?
In addition to design advice from the Biostatistics Core, we received expert guidance from the Ethics and Regulatory Core, who helped us prepare for the central IRB process. The Core’s input was essential to how we developed our protocol’s waiver of informed consent, data handling, and protection of patient privacy. We were able to demonstrate to the IRB that our approach was logical and informed. We think this helped the IRB “get it” and allowed us to more efficiently address patient privacy issues in a vulnerable population across multiple healthcare systems.
What other key challenges have you faced?
One challenge was on the IT side with electronic health record (EHR) integration, which required more customization than we initially planned. How we work with EHR vendors is evolving, and we’ve found good partners so that we can integrate across different systems. This has strengthened our intervention so that it is perceived as more universal than one designed only for a specific EHR system.
Another challenge is the general under-resourcing of healthcare delivery systems for pragmatic research. We found that, regardless of budget, getting approval from system leadership for an IT change is often not enough—what is needed is figuring out who is going to make the change, how much time is involved, and whether the team has the bandwidth to complete the task. You cannot underestimate the degree of difficulty a change poses to a health system that is still struggling to get the clinical side of things right.
The way a study is framed to leadership is important—understand what’s motivating them to participate and move a project forward. With EMBED, we’re trying to take evidence-based research and implement it to improve practice. EMBED is both a research and patient care project. We need to impress upon leadership that we can improve patient outcomes and we’ll pay for it, but we need their help and support in navigating the process through the institution.
What words of advice do you have for investigators conducting their first embedded PCT?
Study teams should think about potential barriers from the beginning and find solutions quickly.
Make sure that health system leadership discusses your project with those on the ground.
Enlist the experts your study needs for each site. In our case, we needed both an IT expert for the operational side and a clinical expert, or we couldn’t have moved the project forward.
Recognize that there are trade-offs in pragmatic design and remember that you’re working with health systems in which your intervention will need to be replicated.
Make your intervention sustainable and easily usable by the clinician, without the need for research or other additional staff.
EMBED is supported within the NIH Collaboratory by a cooperative agreement from the National Institute on Drug Abuse and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about EMBED in the Living Textbook, and learn more about the NIH Collaboratory Trials.
The Collaboratory has made available all the presentations from their recent Steering Committee meeting held in Bethesda May 1-2, 2019.
Highlights of Day 1 included updates on the progress and sustainability of the NIH Collaboratory, perspectives on the landscape of embedded PCTs (ePCTs) and the need for real-world evidence, challenges and lessons learned from the UH3 NIH Collaboratory Trials, updates on progress and transition plans from the UG3 NIH Collaboratory Trials, and discussions on data sharing policy and planning. Day 2 featured an intensive workshop hosted by the NIH with the goal of starting discussions on statistical issues with ePCTs.
An important advantage of embedding pragmatic clinical trials within health care systems is the availability of detailed clinical data on potential participants during trial design. These data can be used to determine eligibility criteria, predict changes in participant characteristics over time, and inform sample size calculations and other design features.
Investigators from the Suicide Prevention Outreach Trial (SPOT), an NIH Collaboratory Trial, recently shared their experiences with using electronic health record data on patients in the participating health systems to inform trial design. The article was published in Clinical Trials.
SPOT was designed to compare the effect of 2 outreach interventions and usual care on the rate of fatal and nonfatal suicide attempts in 3 large health care delivery systems. The investigators used historical data from the electronic health records of the participating health systems to select eligibility requirements, estimate the distribution of patient characteristics during the trial, and calculate statistical power and sample size. Their experiences offer lessons for others who are designing pragmatic trials embedded in health systems with automated data sources.
SPOT was supported within the NIH Collaboratory by a cooperative agreement from the National Institute of Mental Health and received logistical and technical support from the NIH Collaboratory Coordinating Center. Download a study snapshot of SPOT, and learn more about the NIH Collaboratory Trials.
Harriette G.C. Van Spall, MD, MPH, FRCPC
Associate Professor of Medicine
Department of Medicine, Division of Cardiology
Department of Health Research Methods, Evidence, and Impact
McMaster University
Population Health Research Institute
Topic
Adapting Clinical Trial Design to Meet the Needs of Learning Health Systems
Keywords
Learning health system; Pragmatic clinical trial; Patient-Centered Care Transitions in Heart Failure (PACT-HF); Heart failure; Stepped-wedge cluster trial
Key Points
Characteristics of a learning health system include:
Possessing a culture of knowledge and quality improvement
Encouraging research innovation by embedding research into clinical practice and generating knowledge at the point of care
Harnessing data from electronic health records and claims/administrative databases
Fostering trust between research and clinical teams
Engaging patients, clinicians, and key stakeholders
The Patient-Centered Care Transitions in Heart Failure (PACT-HF) trial evaluated the effectiveness of a group of transitional care services in patients hospitalized for HF within a publicly funded healthcare system.
Challenges of a learning health system include integrating care, intervention, and communications across silos; streamlining workflow; preventing “contamination” of usual care; and the limited interoperability of EHRs and slow updates to claims/administrative datasets.
Discussion Themes
Efficacy in explanatory randomized clinical trials (RCTs) does not equate to effectiveness in real-world settings.
Decisions about implementation of an intervention are not made “live”; you must wait until the study has ended, all the data are available for analysis, and analysis is complete before you can inform decision-maker partners about the risks and benefits of the intervention.
JoAnn E. Manson, MD, DrPH
Chief, Division of Preventive Medicine, Brigham and Women’s Hospital
Professor of Medicine and the Michael and Lee Bell Professor of Women’s Health
Harvard Medical School
Professor, Department of Epidemiology
Harvard T.H. Chan School of Public Health
Topic
The VITamin D and OmegA-3 TriaL (VITAL): Design and Results of a Large Pragmatic Trial
The VITAL pragmatic trial evaluated the effects of dietary supplements (vitamin D and omega-3) on reducing risk for developing cancer, heart disease, and stroke in the general population.
Study recruitment involved nationwide and targeted mailings, media reports, advertising, and brochures. Retention included participant newsletters, incentive gifts, and honoraria.
Findings included that neither omega-3s nor vitamin D significantly reduced the primary endpoints of major cardiovascular disease events or total invasive cancer. Omega-3s did reduce total myocardial infarction by 28%, with greatest reductions in those with low dietary fish intake and in African Americans.
Discussion Themes
VITAL’s hybrid design—remote or mail-based intervention plus serial in-clinic visits in a sample—has advantages in promoting quality and cost-efficiency.
Next steps for VITAL include continued follow-up for 5 years; genetic studies; and fostering new ancillary studies through nationwide collaborations.
At the May 2019 meeting of the NIH Collaboratory Steering Committee, we talked with Drs. Ted Melnick and Gail D’Onofrio of