Study design

Toward Causal Inference in Cluster Randomized Trials: Estimands and Reflection on Current Practice

Methods: Minds the Gap Webinar Series
“Toward Causal Inference in Cluster Randomized Trials: Estimands and Reflection on Current Practice”
Fan Li, PhD; Yale School of Public Health
National Institutes of Health, Office of Disease Prevention

Cluster randomized trials (CRTs) involve randomizing groups of individuals to different interventions. While model-based methods are extensively studied for analyzing CRTs, there has been little reflection around the treatment effect estimands at the outset. In the first part of this presentation, we describe two relevant estimands that can be addressed through CRTs and point out that they can differ when the treatment effects vary according to cluster sizes. As a cautionary note, we demonstrate how choices between different analytic approaches can impact the interpretation of results by fundamentally changing the question being asked. In the second part, we revisit the linear mixed model as the most commonly used method for analyzing CRTs. The linear mixed model makes stringent assumptions, including normality, linearity, and typically a compound symmetric correlation structure, all of which may be challenging to verify. However, under certain conditions, we show that the linear mixed model consistently estimates the average causal effect under arbitrary misspecification of its working model. Under equal randomization, its model-based variance estimator, surprisingly, remains consistent under model misspecification, justifying the use of confidence intervals output by standard software. These results hold under both simple and stratified randomization, and serve as an important causal inference justification for linear mixed models. Caveats and extensions of our findings will also be mentioned.

For more information, visit https://prevention.nih.gov/education-training/methods-mind-gap/toward-causal-inference-cluster-randomized-trials-estimands-and-reflection-current-practice.

Grand Rounds August 5, 2022: Economic Evaluation of Platform Trial Designs (Jay JH Park, PhD, MSc)

Posted on by Elizabeth Mccamic

Speaker

Jay JH Park, PhD, MSc
Assistant Professor
Department of Health Research Methods, Evidence and Impact
Faculty of Health Sciences
McMaster University

 

 

Keywords

Platform Trial, Study Design, Pragmatic Clinical Trials

 

Key Points

  • Adaptive trial design is an overarching terminology for trials that use accumulating data in a formal way. In this design, we come up with how we are going to look at the data, how we plan to react to the data, with one or more rounds of internal evaluations or interim reviews where we make the adaptations to the trial design, if the data says we should. The most common types of adaptive trial design are sequential design and response adaptive randomization.
  • Platform trial design refers to trials that are designed with the flexibility to add new intervention. They use a series of documents called “master protocols” that outline trial plans and standard operating procedures for evaluation of multiple interventions. You can conduct platform trials using adaptive trial designs or fixed sample trial designs.
  • To evaluate these trial methods, we did an economic evaluation to determine what are the costs and time requirements conducting a single platform trial versus multiple independent trials? We administered a survey to international experts with publication record on platform trials and master protocols using purposive sampling. The response rate was low (10%). Participants were asked how long it takes and cost for trail set up, conduct, and analysis for a two-arm multi-trial and in addition for a platform trial how long it takes what it costs to add a new intervention.
  • The main outputs were the set-up cost and time, comparing the single study set-up and the total set-up cost and time across different trials and scenarios, and the total cost and time (set-up, conduct, and analysis). We compared a single platform trial to a single 2-arm trial and found it takes considerably less time and cost in setting up a single trial for conventional trials. The findings were similar for cost. There was not much difference in cost between a platform trial and multi-arm trial; setting up a single platform does appear to save money. The single platform trial requires less time measured by total persons.
  • The key takeaway from the simulation is that the platform trial has larger set-up requirements, but it can save money and time in the long run. The platform trial model is not easy but not impossible; as we saw during the COVID-19 pandemic, it can be an effective way to discover important therapies and research in a fast and effective manner.

Discussion Themes

-Did the total cost include both coordinating center costs and site-level costs? We did not get into the site-level costs.

-These platform trials are almost a public good so that there is a sustainable model for the infrastructure so it continues to grow and evolve and no one bears the full cost. Does academic environment create some disincentives as well? Who becomes the PI is an important question because their institution gets the overhead. I’m not sure what the right answer is. There’s lots of red tape in the academic world. It’s an example of the kind of barriers there are to innovative approaches. How can we chip away at the barriers? At the individual level there is interest and commitment, and we see the value in this but there are barriers that do get in the way. Calling them out is step one.

Read the full study.

 

Tags

#pctGR, @Collaboratory1

April 11, 2022: TSOS Implements Suicide Assessment and Monitoring Method in Pragmatic Clinical Trial

Posted on by terren.green@duke.edu

The Trauma Survivors Outcomes and Support (TSOS) trial, an NIH Pragmatic Trials Collaboratory Trial, successfully implemented a large-scale suicide assessment and monitoring method in a pragmatic clinical trial focused on collaborative mental healthcare for traumatic injury survivors. Study intervention and monitoring methods are detailed in a recent publication in Psychiatry.

TSOS researchers analyzed data collected at 25 trauma centers from 635 patients experiencing posttraumatic stress disorder (PTSD) as the result of a traumatic injury. The study used a randomized stepped-wedge design and assigned 370 patients to a control group and 265 to an intervention group.

Patients in the intervention group received proactive injury care management, psychopharmacology, and psychotherapy for PTSD and depression. All patients in both groups were evaluated at 4 timepoints: baseline and 3, 6, and 12 months after injury.

Study personnel interacting with patients participated in a 1-day training workshop to learn study methods and skills for the management of acute suicidal ideation or suicidal intent. Among other measures to assess PTSD symptoms, alcohol use, and physical function, the study team administered the Patient Health Questionnaire (PHQ-9) to screen for suicidal ideation and depression.

Source: Psychiatry. 2022; Spring. doi:10.1080/00332747.2021.1991200.

Patients from both the intervention and control groups who indicated suicidal ideation on the PHQ-9 received calls, texts, and voice messages from study personnel and referral for additional care from a clinician. Study personnel reached out to 161 control and 107 intervention group patients.

The intervention group showed a small but not significant reduction in suicidal ideation compared to the control group.

Lack of a significant treatment effect may be due to the outreach and additional care received by patients in the control group. This level of additional care could be considered a minor intervention for the control group.

Future studies may learn more about treatment differences between control and intervention groups by incorporating implementation process assessments into the design of pragmatic trials.

TSOS was supported within the NIH Collaboratory by a cooperative agreement from the National Institute of Mental Health and by the NIH Common Fund through a cooperative agreement from the Office of Strategic Coordination within the Office of the NIH Director. Learn more about the NIH Collaboratory Trials.

April 1, 2022: ICD-Pieces: Improving Care for CKD, Diabetes and Hypertension in Health Systems (Miguel A. Vazquez, MD; George (Holt) Oliver, MD, PhD)

Posted on by terren.green@duke.edu

Speaker

Miguel A. Vazquez, MD
Professor of Medicine
University of Texas Southwestern Medical Center
Dallas, TX

George (Holt) Oliver, MD, PhD
Vice President Clinical Informatics
Parkland Center for Clinical Innovation
Dallas, TX

Keywords

ICD-Pieces; Chronic kidney disease (CKD); EHR data collection; Diabetes; Hypertension

Key Points

  • ICD-Pieces focuses on the chronic conditions of Diabetes, Hypertension, and CKD. These conditions are common, under-recognized, and can have serious complications.
  • PIECES is an information technology software developed to help facilitate primary care practices provide comprehensive evidence-based care for patients with these chronic conditions.
  • Researchers enrolled 11,000 patients with CKD in the study. The interventions included controlling blood pressure with medication, avoiding hypoglycemia, use of statins, and avoidance of NSAIDs.
  • Interventions were determined to be feasible. Outcomes for study populations are to be determined with further analysis of the data.

Discussion Themes

Pragmatic trials are often practical laboratories for implementation science.

An inherent challenge of collecting data from the EHR record is delay. It may help to have part of the study team embedded in the clinical trial for the data collection aspect, or possibly to collect the data at the time of intervention rather than waiting.

 

Read more about ICD-Pieces.

 

 Tags

#pctGR, @Collaboratory1

March 25, 2022: A Telehealth-Delivered Pragmatic Trial of Mindfulness for Persons with Chronic Low Back Pain (Natalia Morone, MD, MS)

Posted on by terren.green@duke.edu

Speaker

Natalia E. Morone, MD, MS
Associate Professor of Medicine
Boston University/Boston Medical Center

Keywords

Chronic pain; Mindfulness; Stress Reduction; Low back pain; OPTIMUM; The Pain, Enjoyment of Life and General Activity (PEG) Scale; HEAL Initiative

Key Points

  • Chronic back pain is very common across all racial and ethnic groups. Doctors treat chronic back pain with non-pharmacologic methods before resorting to pharmacologic treatments.
  • Optimizing Pain Treatment in Medical Settings Using Mindfulness (OPTIMUM) is a randomized study of mindfulness for low back pain operating in 3 health settings: Boston Medical Center; UPMC, Pittsburgh, PA; and University of North Carolina, Chapel Hill in partnership with Piedmont Health Services.
  • Participation in the OPTIMUM study requires 8 weekly 90 minute group-based sessions of mindfulness meditation training delivered in primary care through a telehealth medical visit with follow up assessments at 6 and 12 months.
  • Four methods of mindfulness meditation are taught: Walking meditation, body scan, breath focused meditation, and mindful stretching.
  • The OPTIMUM study uses the PEG scale as the main outcome measure at the 6 and 12 month follow up assessments.
  • Group tele-health visits provide a variety of benefits for patients including more time with a clinician, better medication adherence, and more patient satisfaction.

Discussion Themes

Primary care usually consists of 1 provider and 1 patient, but family medicine evolved to see patients in a group. This group setting model may have unexpected benefits.

OPTIMUM is part of the PRISM project and is collecting data on participant use of opioids along with all PRISM projects.

Recruiting minority participants has been a priority in the OPTIMUM trial. Targeting recruitment advertisements to zip codes where more minorities live makes a difference in recruiting a diverse population.

 

Read more about the OPTIMUM trial.

 

 Tags

#pctGR, @Collaboratory1

March 11, 2022: Understanding a Patient’s Daily Experience Through Mobile Devices and Wearables: Lessons Learned From the 8,000 Patient MIPACT Study and Implementation in a National Pragmatic Trial (Sachin Kheterpal, MD, MBA; Jessica Golbus, MD; Nicole Pescatore, MPH)

Posted on by terren.green@duke.edu

Speakers

Sachin Kheterpal, MD, MBA
Professor of Anesthesiology
Associate Dean for Research IT
PI for MIPACT and Co-PI for THRIVE University of Michigan

Jessica Golbus, MD
Clinical Instructor, Cardiovascular Medicine
Co-I for MIPACT
University of Michigan

Nicole Eyrich, MPH
MIPACT Clinical Research Project Manager University of Michigan

Keywords

MIPACT; Patient-reported outcomes; Virtual recruitment; Cohort Identification Toolkit; VALENTINE study; THRIVE study; Wearable data research; Propofol

Key Points

  • The MIPACT study combined patient reported outcome data with electronic health record data and data collected from wearable devices.
  • The MIPACT study followed over 7,000 participants for 3 years. Both in-person and virtual recruitment had similar success rates. Participant diversity was a priority during recruitment.
  • The VALENTINE Study was a prospective randomized controlled study using mobile wearable devices to enhance cardiac rehabilitation.
  • Older participants in the VALENTINE study were receptive and capable of using wearable technology to assist data collection for the study.
  • THRIVE is pragmatic clinical trial studying Propofol anesthesia in 22 states and 2 countries.

Discussion Themes

Economic barriers to participation may be present when using wearable device technology as an inclusion criteria for a study. There are various methods to reduce these barriers, such as providing participants with the wearable device and a minimal data plan.

There may be limits to the types of participant reported outcomes that can be collected remotely. Ideally, participants will decide what data researchers will have access to.

Read more about MIPACT, VALENTINE, and THRIVE study.  Read results from MIPACT and VALENTINE.

 

 Tags

#pctGR, @Collaboratory1

February 25, 2022: The Next Generation of Patient-Centered Trials – No Site Visits, Home-delivery of Meds and Patient-reported Outcomes – The CHIEF-HF Trial (John Spertus, MD, MPH, FACC, FAHA)

Posted on by terren.green@duke.edu

Speaker

John Spertus, MD, MPH, FACC, FAHA
Daniel Lauer/Missouri Endowed Chair and Professor
University of Missouri – Kansas City
Clinical Director of Outcomes Research
Saint Luke’s Mid America Heart Institute

Keywords

Heart failure; Canagliflozin; INVOKANA; Kansas City Cardiomyopathy Questionnaire (KCCQ); tele-health; tele-trials

Key Points

  • The CHIEF-HF trial (Canagliflozin: Impact on Health Status, Quality of Life and Functional Status in Heart Failure) was a double blind randomized clinical trial of the medication Canagliflozin for heart failure.
  • CHIEF-HF was designed to learn if patients have fewer symptoms after 3 months of treatment with Canagliflozin.
  • CHIEF-HF used a novel trial design that involved no site-visits and electronic monitoring of patient engagement. The follow up rate for this study was over 97%.
  • The Total Symptom Score on the KC Cardiomyopathy Questionnaire was the key outcome measure in the CHIEF-HF Trial.
  • Heart failure patients treated with Canagliflozin experienced a statistically significant improvements in symptoms.
  • Sites in the study chose recruitment methods that worked best for them. More personalized recruitment strategies were most successful.
  • Difficulties of electronic trials include the technology limitations of the participants, and electronic consent concerns from regulatory agencies. Positives of electronic trials are high enrollment and completion rates.

Discussion Themes

How do technology heavy studies enroll diverse and underserved populations who may not have access to smartphones and wearable technology?

Virtual studies such as this that enrolled at 6 to 8 times the rate on a traditional trial can save quite a bit of money on overhead expenses.

 

Read more about the CHIEF-HF trial.

 

Tags

#pctGR, @Collaboratory1

February 18, 2022: Building a Resource: The Process of Developing a Trans-stakeholder Framework to Enable Pediatric Drug Development (Perdita Taylor-Zapata, MD)

Posted on by terren.green@duke.edu

Speaker

Perdita Taylor-Zapata, MD
Best Pharmaceuticals for Children Act (BPCA) Program Lead and NICHD Program Officer
Obstetric and Pediatric Pharmacology and Therapeutics Branch
National Institute of Child Health and Human Development

Keywords

NIH Best Pharmaceuticals for Children Act; Pediatric Trial Network; Trial design; Pediatric drug development

Key Points

  • The current model for pediatric drug development can be slow and neglect neonates and rare pediatric conditions.
  • The NIH Best Pharmaceuticals for Children Act (BPCA) allows the NIH to conduct clinical trials with off-patent drugs in children.
  • Goals of the BPCA program include developing novel trial designs and including diverse and understudied populations.
  • A new framework to enable pediatric drug development could identify resources to assist in drug development, identify areas in need of further research, provide a pathway for integrating approaches, and connect pediatric researchers.
  • The BPCA went through a rigorous systematic approach to develop a comprehensive resource listing for best practices for pediatric drug trials.

Discussion Themes

Most data collected through the opportunistic model presented is PK data to determine dosing so that a more traditional drug trial can be conducted in the future.

With the right infrastructure in place, such as the Pediatric Trials Network, can substantially improve time to conduct trials.

 

Read more about the BPCA and their commitment to diversity in pediatric drug trials.

 

Tags

#pctGR, @Collaboratory1

February 11, 2022: Great Power and Great Responsibility: Machine Learning in Clinical Research (E. Hope Weissler, MD, MHS; Erich Huang, MD, PhD)

Posted on by terren.green@duke.edu

Speakers

E. Hope Weissler, MD, MHS
Resident, Vascular and Endovascular Surgery
Duke University School of Medicine

Erich Huang, MD, PhD
Chief Science and Innovation Officer, Onduo

Topic

Great Power and Great Responsibility: Machine Learning in Clinical Research

Keywords

Machine Learning; Artificial Intelligence; Data Liquidity; Data Storage; HL7FHIR

Key Points

  • Machine learning may address issues that have reduced the efficiency and effectiveness of clinical research and help clinical research projects reach their full potential.
  • Machine learning may improve the pragmatism of research, decreasing costs and time it takes to conduct a research study.
  • Machine learning can be used to canvas the literature, hypothesize drug-target interactions, propose new therapeutics, and analyze highly dimensional research output.
  • Effects of machine learning are up to us and could potentially reduce the pragmatism of research if applied indiscriminately. Machine learning could produce overly selected study participant groups, too closely managing adherence, and using ultra-high-touch follow-up methods.
  • Data Liquidity refers to the ease with which data can be transferred or exchanged. This depends largely on the manner in which the data is stored.
  • Some forms of data are liquid than others due to privacy, security, and ethical concerns.

Discussion Themes

A lot of emphasis is currently being placed on the mobile/wearable device area, but an equally important area to develop in machine learning is patient identification and recruitment.

Is data ever really de-identified? Should data be owned by the patient? Why is health data treated differently than consumer data? Privacy regulation is difficult and needs to be addressed further by Congress in the future.

 

Read more about Dr. Weissler and Dr. Huang’s machine learning in clinical research.

 

Tags

#pctGR, @Collaboratory1

February 4, 2022: SPIRRIT-HFpEF: Opportunities and Challenges in a Large Registry-Based Randomized Clinical Trial(Adam DeVore, MD, MHS; Lars Lund, MD, PhD)

Posted on by terren.green@duke.edu

Speakers

Adam DeVore, MD, MHS
Associate Professor of Medicine
Duke University Medical Center
Duke Clinical Research Institute

Lars Lund, MD, PhD
Professor of Cardiology
Karolinska Institutet
Karolinska University Hospital

Topic

SPIRRIT-HFpEF: Opportunities and Challenges in a Large Registry-based Randomized Clinical Trial

Keywords

Heart Failure; SPIRRIT-HFpEF; Randomized clinical trial; Spironolactone; Eplerenone; Swedish Heart Failure Registry (SwedeHF)

Key Points

  • The SPIRRIT-HFpEF trial, conducted Sweden and the US, was a randomized pragmatic clinical trial of spironolactone or eplerenone in heart failure.
  • Death from heart disease is decreasing while death from Heart Failure is increasing.
  • The SPIRRIT-HFpEF trial focused on improving the trajectory for the growing heart failure population.
  • Patients treated with Spironolactone had a modest but not statistically significant improvement over placebo, but total hospitalizations were less.
  • Patients with a lower ejection fraction were more likely to benefit than patients with a higher ejection fraction.
  • The Swedish Heart Failure Registry (SwedeHF) has been collecting data from HF patients since 2000.’

Discussion Themes

The hardest aspect of a clinical trial is recruitment and enrollment. Patients are spread out over the health care system. The challenge is getting staff and personnel to do the work of screening and prescreening.

In the SPIRRIT-HFpEF, the drawbacks of not blinding were small and the costs of blinding would have been huge.

 

Learn more about the SPIRRIT-HFpEF trial and the Swedish Heart Failure Registry.  Read about the SPIRRIT-HFpEF trial results.

 

Tags

#pctGR, @Collaboratory1