November 13, 2018: Summary of Workshop on Pragmatic Trials of Therapeutic A vs B Interventions Now Available

The NIH Collaboratory recently convened a workshop to explore embedded pragmatic clinical trials comparing two or more therapeutic medical interventions. These “A vs B” trials are meant to test existing, viable treatment alternatives where there is uncertainty about which treatment is best in which populations. There are unique barriers that make these types of pragmatic trials especially challenging to implement. For the workshop, a panel of experts gathered to discuss challenges and solutions regarding partnering with healthcare systems to conduct the trials, unique legal and ethical issues, and design and operational considerations. The summary of the workshop is now available: Workshop Summary: Embedded Pragmatic Clinical Trials of Therapeutic A vs. B Interventions

 

Additional Resources:

Embedded pragmatic clinical trials of therapeutic A vs. B interventions workshop videocast.

 

October 1, 2018: Dr. Greg Simon Uses a Pie Eating Contest Analogy to Explain the Intraclass Correlation Coefficient

In a new video, Dr. Greg Simon explains the intraclass correlation coefficient (ICC) with an analogy to a pie eating contest. The ICC is a descriptive statistic that measures the correlations among members of a group, and it is an important tool for cluster-randomized pragmatic trials because this calculation helps determine the sample size needed to detect an effect.

Greg Simon from NIH Collaboratory on Vimeo.

“When we randomize treatments by doctors, clinics, or even whole health systems, we need to think about how things cluster, and the intraclass correlation coefficient is the measure of that clustering. When we think about sample sizes in pragmatic clinical trials, it’s important to understand what an intraclass correlation coefficient actually is.”

For most pragmatic trials, the ICC will be between 0 and 1. If the outcomes in a group are completely correlated (ICC=1), then all participants within the group are likely to have the same outcome. When ICC=1, sampling one participant from the cluster is as informative as sampling the whole cluster, and many clusters will be needed to detect an effect. If there is no correlation among members of the groups (ICC=0), then the available sample size for the study is essentially the number of participants.

For more on the ICC, see the Intraclass Correlation section in the Living Textbook or this working document from the Collaboratory’s Biostatistics and Study Design Core.

April 9, 2018: PPACT Study Design Paper Published

Congratulations to Dr. Lynn DeBar and the investigators of the Collaborative Care for Chronic Pain in Primary Care pragmatic trial for recently publishing their study design paper. One of the NIH Collaboratory Demonstration Projects, the trial is designed to test whether a primary care–based behavioral intervention the Pain Program for Active Coping and Training (PPACT)—will provide a “more effective, safer, and more satisfactory alternative to opioid-based chronic pain treatment” than usual care for patients on chronic opioid treatment (Debar et al 2018). Learn more about this innovative trial in the article in Contemporary Clinical Trials. You can also download a trial snapshot.

Full Citation: DeBar L, Benes L, Bonifay A, et al. Interdisciplinary team-based care for patients with chronic pain on long-term opioid treatment in primary care (PPACT) – Protocol for a pragmatic cluster randomized trial. Contemporary Clinical Trials. 2018;67:91-99. doi:10.1016/j.cct.2018.02.015

January 19, 2018: New Research Methods Resources Website on Group- or Cluster-Randomized Studies

The National Institutes of Health (NIH) Office of Extramural Research has released new clinical trial requirements for grant applications and contract proposals due on or after January 25, 2018. In anticipation of these new requirements, the NIH modified the Application Guide and the Review Criteria to address methodological problems common to many clinical trials. As group- or cluster-randomization designs are increasingly common in both basic and applied research, the new Application Guide includes links to the new Research Methods Resources website, which provides resources for investigators considering these group- or cluster-randomized designs, including lists of NIH webinars, key references, and statements to help investigators prepare sound applications and avoid methodological pitfalls.

NIH & FDA seek feedback on new clinical trial protocol template


As part of their ongoing effort to improve the speed and efficiency of conducting clinical trials, the NIH-FDA Joint Leadership Council has created a draft clinical trial protocol template. The template contains instructional and sample text intended to assist NIH-funded investigators in writing protocols for phase 2 or 3 clinical trials that require Investigational New Drug (IND) or Investigational Device Exemption (IDE) applications. Feedback is sought from investigators, investigator-sponsors, institutional review board members, and other stakeholders involved in protocol development and review.

Our goal is to provide an organized way for creative investigators to describe their plans so that others can understand them. – Dr. Pamela McInnes, NIH

Details on the rationale and development of the protocol template are on these blog posts:

Notice Number: NOT-OD-16-043. Responses accepted through April 17, 2016.

You can access the template document as well as a template shell, comment form, and other resources at NIH’s Clinical Research Policy website.

STOP CRC Trial: Analytic Challenges and Pragmatic Solutions


Investigators from the STOP CRC pragmatic trial, an NIH Collaboratory Demonstration Project, have recently published an article in the journal eGEMs describing solutions to issues that arose in the trial’s implementation phase. STOP CRC tests a program to improve colorectal cancer screening rates in a collaborative network of Federally Qualified Health Centers by mailing fecal immunochemical testing (FIT) kits to screen-eligible patients at clinics in the intervention arm. Clinics in the control arm provided opportunistic colorectal-cancer screening to patients at clinic visits in Year 1 and implemented the intervention in Year 2. In this cluster-randomized trial, clinics are the unit of analysis, rather than individual patients, with the primary outcome being the proportion of screen-eligible patients at each clinic who complete a FIT.

The team dealt with various challenges that threatened the validity of their primary analysis, one of which related to potential contamination of the primary outcome due to the timing of the intervention rollout: for control participants, the Year 2 intervention actively overlapped with the Year 1 control measurements. The other challenge was due to a lack of synchronization between the measurement and accrual windows. To deal with these issues, the team had to slightly modify the study design in addition to developing a few sensitivity analyses to better estimate the true impact of the intervention.

“While the nature of the challenges we encountered are not unique to pragmatic trials, we believe they are likely to be more common in such trials due to both the types of designs commonly used in such studies and the challenges of implementing system-based interventions within freestanding health clinics.” (Vollmer et al. eGEMs 2015)

The Publish EDM Forum Community publishes eGEMs (generating evidence & methods to improve patient outcomes) and provides free and open access to this methods case study. Readers can access the article here.

Findings from STOP CRC on Pragmatic Trial Recruitment


Gloria Coronado, PhD, and Beverly Green, MD, MPH, Principal Investigators, STOP CRC Trial
Gloria Coronado, PhD, and Beverly Green, MD, MPH, Principal Investigators, STOP CRC Trial

Drs. Beverly Green and Gloria Coronado and colleagues have published an article in Clinical Trials describing the challenges of recruiting participants into large, multisite pragmatic clinical trials—particularly at the health system level. STOP CRC is one of the NIH Collaboratory’s pragmatic clinical trial Demonstration Projects, which are intended to provide a framework of implementation methods and best practices to enable participation of varied health care systems in clinical research.

STOP CRC is testing a culturally tailored, health care system–based program to improve colorectal cancer screening rates in a community-based collaborative network of federally qualified health centers. The authors observed that recruiting sites to participate in pragmatic trials is time-intensive and involves both preparing materials and organizing face-to-face meetings with staff and clinic leaders. Yet little is known about the characteristics of nonparticipating sites and clinic-level factors that may influence willingness to participate in a pragmatic trial.

“Our findings underscore the importance of assessing and reporting recruitment success at the organizational and/or clinic level in order to know the external validity of the findings and may inform future efforts to select and recruit health systems to participate in pragmatic research.” (Coronado, et al. Clin Trials 2015)


Designing Fit-for-Purpose Trials with PRECIS-2


Few clinical trials are entirely explanatory (done in an idealized setting) or entirely pragmatic (done in a usual-care setting); rather, trials are situated somewhere along a continuum of applicability. Pragmatic clinical trials are trials designed with pragmatic qualities and are intended to inform decision makers, including patients, clinicians, administrators, and policymakers, about the relative benefits, burdens, and risks of a health intervention.

To help trialists assess how closely their trial’s design matches its intended purpose, a group of trialists and methodologists developed a design tool, the Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS. Originally implemented in 2008, the wheel-shaped indicator tool recently underwent a revision, leading to PRECIS-2. The revised, validated tool guides trialists to prospectively consider the design of their trial along 9 domains: eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.

PRECIS-2 Wheel
PRECIS-2 Wheel*

*Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.

See the PRECIS-2 introductory video (YouTube) and below links for detailed user information.

Health Informatics Centre website
PRECIS-2 Toolkit
Wheel examples

Cluster Randomized Trial Design Featured in JAMA’s Guide to Statistics and Methods Series


A new article published this week in JAMA describes the cluster randomized trial design. The article is part of JAMA’s Guide to Statistics and Methods series, which publishes explanations of analytic and methodologic approaches used in current research articles to help clinicians better understand the research.

In “Cluster Randomized Trials: Evaluating Treatments Applied to Groups,” Drs. William J. Meurer and Roger J. Lewis define cluster randomization, describe its advantages and limitations, and provide guidance on interpreting cluster randomized trials. The article discusses aspects of a recent cluster randomized trial, the RESTORE trial, as an example.

In RESTORE, pediatric intensive care units were randomized to assess the effects of a nurse-implemented sedation protocol for children with acute respiratory failure on mechanical ventilation. As Meurer and Lewis point out, “interventions that involve training multidisciplinary health care teams are practically difficult to conduct using individual-level randomization, as health care practitioners cannot easily unlearn a new way of taking care of patients.” Cluster randomized designs are therefore often used for this type of research, and it is important for clinicians to be able to understand and evaluate these studies.


Reference:
Meurer WJ, Lewis RJ. Cluster randomized trials: evaluating treatments applied to groups. JAMA. 2015;313:2068-2069. PMID: 26010636. doi:10.1001/jama.2015.5199.

Collaboratory Biostatistics and Study Design Core Releases Guidance Documents


The NIH Collaboratory’s Biostatistics and Study Design Core has released the first in a series of guidance documents focusing on statistical design issues for pragmatic clinical trials. Each of the four guidance documents are intended to help researchers by providing a synthesis of current developments in the field, discuss possible future directions, and, where appropriate, make recommendations for application to pragmatic clinical research.

The guidance documents are available through the Living Textbook and can be accessed on the “Tools for Research” tab or directly here.