August 5, 2019: New Section of Living Textbook Addresses Missing Data in Intention-to-Treat Analyses

A new section of the NIH Collaboratory’s Living Textbook of Pragmatic Clinical Trials discusses challenges associated with missing data that result from noncompliance, crossover, and dropout.

Many randomized controlled trials use an intention-to-treat (ITT) analysis to measure the real-world effects of the intervention. The newly published section, Missing Data and Intention-to-Treat Analyses, considers the population-level causal effects in these trials when there is noncompliance or missing outcome data.

“One rationale for the ITT approach is that it evaluates the real-world effects of the intervention. However, a common misconception is that the ITT analysis will be unbiased regardless of crossover or missing data.”

The new section also introduces a white paper from the NIH Collaboratory’s Biostatistics and Study Design Core, “Analyses of Randomized Controlled Trials in the Presence of Noncompliance and Study Dropout.” This working document offers analysts a more detailed discussion of treatment effects in ITT analyses, including a case example and recommended strategies for estimating and reporting both ITT effects and average causal effects.

The Biostatistics and Study Design Core works with the NIH Collaboratory Trial teams to create guidance and technical documents regarding study design and biostatistical issues relevant to pragmatic clinical trials.

August 2, 2019: AI and the Future of Psychiatry (Murali Doraiswamy, MBBS)

Speaker

Murali Doraiswamy, MBBS
Professor of Psychiatry and Behavioral Sciences
Duke School of Medicine

Topic

AI and the Future of Psychiatry

Keywords

Artificial intelligence; Machine learning; Psychiatry; Ethical adoption of technologies; Mental health; Wearables; Mobile health

Key Points

  • There is growing evidence from randomized controlled trials of the efficacy of using digital tools in mental health diagnosis and treatment.
  • Could artificial intelligence (AI) and machine learning technologies be used to:
    • Reduce the stigma associated with mental health treatment?
    • Predict the risk for future suicide?
    • Detect Alzheimer’s years before diagnosis?
  • Categories of AI applications include low-risk apps that measure but do not diagnose, and apps used in diagnosis or treatment that must meet the same high standards of evidence as medications.
  • Clinicians still struggle with how to integrate patient data from wearable devices. AI technology might help if it could be used to synthesize the data into a risk profile for an individual.

Discussion Themes

What are the roles of stress, exercise, and sleep in mental health, and can autonomic data from wearables help explain the variance in mental health symptoms?

To develop evidence thresholds for AI, we need larger scale public-private partnerships as well as pragmatic trials addressing key clinical questions.

Read more from Dr. Doraiswamy in How to Use Technology Ethically to Increase Access to Mental Healthcare.
Tags

#AI, #pctGR, @Collaboratory1

June 20, 2019: EMBED Investigators Discuss Progress and Transition to Implementation Phase

At the May 2019 meeting of the NIH Collaboratory Steering Committee, we talked with Drs. Ted Melnick and Gail D’Onofrio of EMBED, an NIH Collaboratory Trial, to hear about progress and challenges during the UG3 planning phase. The goal of EMBED is to test whether implementation of a user-centered clinical decision support system increases adoption of initiation of buprenorphine/naloxone into the routine emergency care of patients with opioid use disorder. In the UG3 phase, the study team put in place the infrastructure of a pragmatic, multicenter, parallel, group-randomized health IT intervention. EMBED recently transitioned to the UH3 implementation phase and plans to launch the intervention at 20 sites across 5 healthcare systems in August 2019.

“With EMBED, we’re trying to take evidence-based research and implement it to improve practice. EMBED is both a research and patient care project.”

Were there any surprises during the study’s planning phase?

The first surprise came at last year’s Steering Committee meeting, when we met with the Biostatistics and Study Design Core. They encouraged us to change our original study design from stepped-wedge to group-randomized, which we did. We think this advice led to a stronger study. The main reason for this is the group-randomized design’s ability to better account for temporal changes. Since our intervention is being conducted in the middle of an opioid crisis, there are potentially other concurrent interventions that could make it difficult to determine the effect of our intervention. The group-randomized design should give us better insight into whether our intervention is driving behavior change in treating patients with opioid use disorder.

What is an example of a challenge that you were able to overcome with the help of a Core Working Group?

In addition to design advice from the Biostatistics Core, we received expert guidance from the Ethics and Regulatory Core, who helped us prepare for the central IRB process. The Core’s input was essential to how we developed our protocol’s waiver of informed consent, data handling, and protection of patient privacy. We were able to demonstrate to the IRB that our approach was logical and informed. We think this helped the IRB “get it” and allowed us to more efficiently address patient privacy issues in a vulnerable population across multiple healthcare systems.

What other key challenges have you faced?

One challenge was on the IT side with electronic health record (EHR) integration, which required more customization than we initially planned. How we work with EHR vendors is evolving, and we’ve found good partners so that we can integrate across different systems. This has strengthened our intervention so that it is perceived as more universal than one designed only for a specific EHR system.

Another challenge is the general under-resourcing of healthcare delivery systems for pragmatic research. We found that, regardless of budget, getting approval from system leadership for an IT change is often not enough—what is needed is figuring out who is going to make the change, how much time is involved, and whether the team has the bandwidth to complete the task. You cannot underestimate the degree of difficulty a change poses to a health system that is still struggling to get the clinical side of things right.

The way a study is framed to leadership is important—understand what’s motivating them to participate and move a project forward. With EMBED, we’re trying to take evidence-based research and implement it to improve practice. EMBED is both a research and patient care project. We need to impress upon leadership that we can improve patient outcomes and we’ll pay for it, but we need their help and support in navigating the process through the institution.

What words of advice do you have for investigators conducting their first embedded PCT?

  • Study teams should think about potential barriers from the beginning and find solutions quickly.
  • Make sure that health system leadership discusses your project with those on the ground.
  • Enlist the experts your study needs for each site. In our case, we needed both an IT expert for the operational side and a clinical expert, or we couldn’t have moved the project forward.
  • Recognize that there are trade-offs in pragmatic design and remember that you’re working with health systems in which your intervention will need to be replicated.
  • Make your intervention sustainable and easily usable by the clinician, without the need for research or other additional staff.

EMBED is supported within the NIH Collaboratory by a cooperative agreement from the National Institute on Drug Abuse and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about EMBED in the Living Textbook, and learn more about the NIH Collaboratory Trials.

June 7, 2019: Meeting Materials from the 2019 NIH Collaboratory Steering Committee Meeting

The Collaboratory has made available all the presentations from their recent Steering Committee meeting held in Bethesda May 1-2, 2019.

Highlights of Day 1 included updates on the progress and sustainability of the NIH Collaboratory, perspectives on the landscape of embedded PCTs (ePCTs) and the need for real-world evidence, challenges and lessons learned from the UH3 NIH Collaboratory Trials, updates on progress and transition plans from the UG3 NIH Collaboratory Trials, and discussions on data sharing policy and planning. Day 2 featured an intensive workshop hosted by the NIH with the goal of starting discussions on statistical issues with ePCTs.

View or download the meeting materials on the website.

June 3, 2019: SPOT Illustrates Use of Real-World Health System Data in Designing Embedded Pragmatic Clinical Trials

An important advantage of embedding pragmatic clinical trials within health care systems is the availability of detailed clinical data on potential participants during trial design. These data can be used to determine eligibility criteria, predict changes in participant characteristics over time, and inform sample size calculations and other design features.

Investigators from the Suicide Prevention Outreach Trial (SPOT), an NIH Collaboratory Trial, recently shared their experiences with using electronic health record data on patients in the participating health systems to inform trial design. The article was published in Clinical Trials.

SPOT was designed to compare the effect of 2 outreach interventions and usual care on the rate of fatal and nonfatal suicide attempts in 3 large health care delivery systems. The investigators used historical data from the electronic health records of the participating health systems to select eligibility requirements, estimate the distribution of patient characteristics during the trial, and calculate statistical power and sample size. Their experiences offer lessons for others who are designing pragmatic trials embedded in health systems with automated data sources.

SPOT was supported within the NIH Collaboratory by a cooperative agreement from the National Institute of Mental Health and received logistical and technical support from the NIH Collaboratory Coordinating Center. Download a study snapshot of SPOT, and learn more about the NIH Collaboratory Trials.

May 31, 2019: Adapting Clinical Trial Design to Meet the Needs of Learning Health Systems (Harriette Van Spall, MD, MPH)

Speaker

Harriette G.C. Van Spall, MD, MPH, FRCPC
Associate Professor of Medicine
Department of Medicine, Division of Cardiology
Department of Health Research Methods, Evidence, and Impact
McMaster University
Population Health Research Institute

Topic

Adapting Clinical Trial Design to Meet the Needs of Learning Health Systems

Keywords

Learning health system; Pragmatic clinical trial; Patient-Centered Care Transitions in Heart Failure (PACT-HF); Heart failure; Stepped-wedge cluster trial

Key Points

  • Characteristics of a learning health system include:
    • Possessing a culture of knowledge and quality improvement
    • Encouraging research innovation by embedding research into clinical practice and generating knowledge at the point of care
    • Harnessing data from electronic health records and claims/administrative databases
    • Fostering trust between research and clinical teams
    • Engaging patients, clinicians, and key stakeholders
  • The Patient-Centered Care Transitions in Heart Failure (PACT-HF) trial evaluated the effectiveness of a group of transitional care services in patients hospitalized for HF within a publicly funded healthcare system.
  • Challenges of a learning health system include integrating care, intervention, and communications across silos; streamlining workflow; preventing “contamination” of usual care; and the limited interoperability of EHRs and slow updates to claims/administrative datasets.

Discussion Themes

Efficacy in explanatory randomized clinical trials (RCTs) does not equate to effectiveness in real-world settings.

Decisions about implementation of an intervention are not made “live”; you must wait until the study has ended, all the data are available for analysis, and analysis is complete before you can inform decision-maker partners about the risks and benefits of the intervention.

Read more about the PACT-HF study and results in JAMA Network (Van Spall et al. 2019)

Tags

#pctGR, @Collaboratory1

May 17, 2019: The VITamin D and OmegA-3 TriaL (VITAL): Design and Results of a Large Pragmatic Trial (JoAnn E. Manson, MD, DrPH)

Speaker

JoAnn E. Manson, MD, DrPH
Chief, Division of Preventive Medicine, Brigham and Women’s Hospital
Professor of Medicine and the Michael and Lee Bell Professor of Women’s Health
Harvard Medical School
Professor, Department of Epidemiology
Harvard T.H. Chan School of Public Health

Topic

The VITamin D and OmegA-3 TriaL (VITAL): Design and Results of a Large Pragmatic Trial

Keywords

Pragmatic clinical trial; Dietary supplements; Primary prevention; Mail-based randomized clinical trial; Cancer prevention; Cardiovascular disease prevention; vitamin D; Omega-3 fatty acids

Key Points

  • The VITAL pragmatic trial evaluated the effects of dietary supplements (vitamin D and omega-3) on reducing risk for developing cancer, heart disease, and stroke in the general population.
  • Study recruitment involved nationwide and targeted mailings, media reports, advertising, and brochures. Retention included participant newsletters, incentive gifts, and honoraria.
  • Findings included that neither omega-3s nor vitamin D significantly reduced the primary endpoints of major cardiovascular disease events or total invasive cancer. Omega-3s did reduce total myocardial infarction by 28%, with greatest reductions in those with low dietary fish intake and in African Americans.

Discussion Themes

VITAL’s hybrid design—remote or mail-based intervention plus serial in-clinic visits in a sample—has advantages in promoting quality and cost-efficiency.

Next steps for VITAL include continued follow-up for 5 years; genetic studies; and fostering new ancillary studies through nationwide collaborations.  

Visit the VITAL study website and read more about the results of VITAL (Manson et al., New Engl J Med, 2019)

Tags

#dietarysupplements, #pctGR, @Collaboratory1

May 10, 2019: Treating Data as an Asset: Data Entrepreneurship in the Service of Patients (Eric Perakslis, PhD)

Speaker

Eric D. Perakslis MS, PhD
Rubenstein Fellow, Duke University
Lecturer, Department of Biomedical Informatics
Harvard Medical School

Topic

Treating Data as an Asset: Data Entrepreneurship in the Service of Patients

Keywords

Digital health; Health data; General Data Protection Regulation (GDPR); Data sharing

Key Points

  • The only 100% common element of digital transformation across all industries is data.
  • With data and digital transformation, patients are changing: They are active, connected, informed, and savvy.
  • Security, compliance, and privacy are different things.

Discussion Themes

Is there any hope of data sharing policies helping to bridge the micro and macro silos of healthcare data?

As data starts to flows through institutions, it ends up in multiple places. Part of sharing data is protecting a single source of truth.

If something is relevant to the bedside, it’s worth doing.

Read Dr. Perakslis’s commentary in The Lancet (May 2019).

Tags

#healthdata, #pctGR, @Collaboratory1

April 26, 2019: The VERITAS Trial: Virtual Exercise Rehabilitation at the Intersection of Evidence, Implementation, and Policy (Janet Prvu Bettger, ScD)

Speaker

Janet Prvu Bettger, ScD, FAHA
Associate Professor
Duke Department of Orthopaedic Surgery
Duke Clinical Research Institute

Topic

The VERITAS Trial: Virtual Exercise Rehabilitation at the Intersection of Evidence, Implementation, and Policy

Keywords

Rehabilitation; Virtual physical therapy; Patient outcomes; Physical therapy; Orthopaedic surgery; Total knee replacement; Digital technology; Telehealth

Key Points

  • The VERITAS trial evaluated the effects of physical therapy–supported virtual exercise compared with traditional home- or clinic-based physical therapy after total knee replacement. Outcome measures included 90-day health service use costs; patient-centered outcomes; and differential improvement from 6 weeks to 3 months.
  • The Center for Connected Health Policy found that while most states currently have established telehealth policies for primary care providers, these often do not include physical or occupational therapists.
  • Tele-rehabilitation facilitates communication between the patient and physical therapist in real time. The VERA™ technology provides a virtual physical therapist assistant for patients and clinicians, offering a digital interface that includes patient education, longitudinal functional assessments, telehealth video conferencing, personalized exercises, and remote monitoring of patient progress.

Discussion Themes

Study results support effectiveness and safety hypotheses: that tele-rehabilitation is noninferior to traditional physical therapy with respect to range of motion, walking speed, pain, or rehospitalization. However, it was not shown that the intervention is noninferior with respect to falls after hospital discharge.

Because virtual physical therapy interventions can save total costs, prevent readmissions, and improve mobility, it will be important to expand access to tele-rehabilitation and to advance policies that include physical therapists.

Read more about the VERITAS project and at ClinicalTrials.gov.

Tags

#telehealth, #pctGR, @Collaboratory1

April 19, 2019: Trauma Survivors Outcomes & Support (TSOS) Pragmatic Trial: Revisiting Effectiveness & Implementation Aims (Doug Zatzick, MD)

Speaker

Doug Zatzick, MD
Professor of Psychiatry
Harborview Medical Center
University of Washington School of Medicine

Topic

Trauma Survivors Outcomes & Support (TSOS) Pragmatic Trial: Revisiting Effectiveness & Implementation Aims

Keywords

Trauma outcomes; NIH Collaboratory Trial; Hybrid study design; Implementation science; Pragmatic clinical trial; Cluster randomization; Stepped-wedge design; Posttraumatic stress disorder; PRECIS-2; Mental health intervention

Key Points

  • The TSOS NIH Collaboratory Trial is a cluster-randomized, stepped-wedge trial conducted at 25 U.S. trauma centers. The intervention involves an electronic health record PTSD screen and a baseline PTSD and comorbidity assessment. TSOS is turned on at each site across 4 “waves.”
  • During the course of this hybrid effectiveness-implementation trial, two domains on PRECIS-2 (Pragmatic-Explanatory Continuum Indicator Summary) were scored as more pragmatic and one domain as more explanatory than at the outset of the study.
  • The study team developed a methodology for assessing TSOS implementation aims. Called RAPICE (Rapid Assessment Procedure Informed Clinical Ethnography), the method yielded findings around recurrent intervention and research staff turnover across sites; observations that some patients do not engage in the intervention; and ways to inform a priori secondary hypotheses that suggest per-protocol modifications to the original intention-to-treat analyses.
  • TSOS will present results at the 2020 summit of the American College of Surgeons with the potential to integrate findings into the College’s regulatory and verification processes.

Discussion Themes

Regarding the need to collect outcome data, there may be an important distinction between two aspects of “pragmatic.” That is, while collecting outcome data makes a trial more expensive (one aspect of pragmatic), it doesn’t necessarily affect relevance or generalizability (another, more important, aspect of pragmatic).

Might there be studies which, by design, are not aiming to be on the outer [more pragmatic] spokes of the PRECIS-2 wheel?

Read more about the TSOS NIH Collaboratory Trial.

Tags

#pctGR, @Collaboratory1, @PRECIS_2