In a new video in the Living Textbook, Dr. Greg Simon describes the differences between individual, cluster, and stepped-wedge randomization using props, including marbles, Play-Doh, and glassware.
“In the end, it’s all about randomly assigning who gets which treatment, or who gets which treatment when, so that we’re able to make some un-biased judgement about which treatment is really better.” —Greg Simon, MD
As part of an article published in Annals of Internal Medicine, Dr. Greg Simon created a short video in which he describes concerns related to data sharing and embedded research, as well as potential solutions for those concerns. We recently added this video to the Living Textbook chapter on Data Sharing and Embedded Research. In the chapter, the authors expand on the ideas presented in the Annals article and fame them using lessons learned from the NIH Collaboratory Trials. Data collected as part of research embedded in a health system comes from a fundamentally different context than stand-alone explanatory trials. When they are taken out of context or used for comparisons, they have the potential to do harm—something that can potentially discourage health systems from volunteering to participate in embedded research. The authors suggest that data sharing plans for embedded research be developed in partnership with health system leaders in ways that maximize the amount of data that can be shared while protecting patient privacy and healthcare system interests.
“Ultimately, it’s a practical question: if we want healthcare providers and healthcare systems to participate in research, we shouldn’t expect them to bear extra risk. In an ideal world, all information about the quality of health care and healthcare outcomes across the country would be completely open to everyone, but we don’t live in that world now. So if we are asking healthcare providers and healthcare systems to open up and be more transparent by participating in research, we certainly would not want to punish those who volunteer.” — Simon et al. in video for Ann Intern Med
Simon G, Coronado G, DeBar L, et al. Data Sharing and Embedded Research: Introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: https://rethinkingclinicaltrials.org/data-share-top/data-sharing-and-embedded-research-introduction/. Updated November 13, 2017.
In this interview, Dr. Miguel Vazquez gives an update on the first years of the Improving Chronic Disease Management with Pieces (ICD-Pieces) trial. Dr. Vazquez discussed the status of his trial, challenges and surprises, and advice he has for new investigators.
“Try to really learn from others who have done this—even if you are the first one doing your specific trial with your specific questions. It was helpful for us to learn from the other Collaboratory projects; they had already faced some problems, and we were able to anticipate and develop solutions proactively.” – Dr. Miguel Vazquez
Read more from Dr. Vazquez in the full interview (PDF).
In this interview, Dr. Doug Zatzick gives an update on the first years of the Trauma Survivors Outcomes and Support (TSOS) trial. Dr. Zatzick discussed the status of his trial, challenges and surprises, and advice he has for new investigators.
Dr. Zatzick’s advice: “Embed implementation teams within embedded trials. The bottom line is, go to the sites, do training at the sites and with the team, and take field notes in real time. ”
Read more from Dr. Zatzick in the full interview.
In this interview, Dr. Vincent Mor gives an update on the first years of the Pragmatic Trial of Video Education in Nursing Homes (PROVEN) project. Dr. Mor discussed the status of his trial, challenges and surprises, and advice he has for new investigators.
Dr. Mor’s advice: “The health care system must agree that the outcome your intervention is seeking to achieve is consistent with their mission. Your outcome goal should be something they care about.”
Read more from Dr. Mor in the full interview.
Investigators from the STOP CRC pragmatic trial, an NIH Collaboratory Trial, have recently published an article in the journal eGEMs describing solutions to issues that arose in the trial’s implementation phase. STOP CRC tests a program to improve colorectal cancer screening rates in a collaborative network of Federally Qualified Health Centers by mailing fecal immunochemical testing (FIT) kits to screen-eligible patients at clinics in the intervention arm. Clinics in the control arm provided opportunistic colorectal-cancer screening to patients at clinic visits in Year 1 and implemented the intervention in Year 2. In this cluster-randomized trial, clinics are the unit of analysis, rather than individual patients, with the primary outcome being the proportion of screen-eligible patients at each clinic who complete a FIT.
The team dealt with various challenges that threatened the validity of their primary analysis, one of which related to potential contamination of the primary outcome due to the timing of the intervention rollout: for control participants, the Year 2 intervention actively overlapped with the Year 1 control measurements. The other challenge was due to a lack of synchronization between the measurement and accrual windows. To deal with these issues, the team had to slightly modify the study design in addition to developing a few sensitivity analyses to better estimate the true impact of the intervention.
“While the nature of the challenges we encountered are not unique to pragmatic trials, we believe they are likely to be more common in such trials due to both the types of designs commonly used in such studies and the challenges of implementing system-based interventions within freestanding health clinics.” (Vollmer et al. eGEMs 2015)
The Publish EDM Forum Community publishes eGEMs (generating evidence & methods to improve patient outcomes) and provides free and open access to this methods case study. Readers can access the article here.
A new article published in the journal Trials provides a look at how the Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS, rating system can be applied to clinical trials designs in order to examine where a given study sits on the spectrum of explanatory versus pragmatic clinical trials.
The PRECIS-2 criteria are used to rate study designs as more or less “pragmatic” according to multiple domains that include participant eligibility, recruitment methods, setting, organization, analysis methods, primary outcomes, and more. In this context, “pragmatic” refers to trials that are designed to study a therapy or intervention in a “real world” setting similar or identical to the one in which the therapy will actually be used. Pragmatic trials stand in contrast to explanatory trials, which are typically designed to demonstrate the safety and efficacy of an intervention under highly controlled conditions and in carefully selected groups of participants, but which may also be difficult to generalize to larger or more varied populations.
Clinical trials are almost never wholly “explanatory” or wholly “pragmatic.” Instead, many studies exist somewhere on a spectrum between these two categories. However, understanding how these different attributes apply to trials can help researchers design studies that are optimally fit for purpose, whether that purpose is to describe a biological mechanism (as in an explanatory trial) or to show how effective an intervention is when used across a broad population of patients (as in a pragmatic trial).
In their article in Trials, authors Karin Johnson, Gila Neta, and colleagues applied PRECIS-2 criteria to 5 pragmatic clinical trials (PCTs) being conducted through the NIH Collaboratory. Each trial was found to rate as “highly pragmatic” across the multiple PRECIS-2 domains, highlighting the tool’s potential usefulness in guiding decisions about study design, but also revealing a number of challenges in applying it and interpreting the results.
Study authors Johnson and Neta will be discussing their findings during the NIH Collaboratory’s Grand Rounds on Friday, January 22, 2016 (an archived version of the presentation will be available the following week).
Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, Stoney CM, Farrell MM, Glasgow RE. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32. doi: 10.1186/s13063-016-1158-y. PMID: 26772801. PMCID: PMC4715340.
You can read more about the NIH Collaboratory PCTs featured as part of this project at the following links: ABATE (Active Bathing to Eliminate Infection) LIRE (A pragmatic trial of Lumbar Image Reporting with Epidemiology) PPACT (Collaborative Care for Chronic Pain in Primary Care) STOP-CRC (Strategies & Opportunities to Stop Colon Cancer in Priority Populations) TIME (Time to Reduce Mortality in End-Stage Renal Disease)
Additional Resources An introductory slide set on PCTs (by study author Karin Johnson) is available from the Living Textbook: Introduction to Pragmatic Clinical Trials The University of Colorado Denver - Anschutz Medical Campus publishes an electronic textbook on pragmatic trials: Pragmatic Trials: A workshop Handbook
The NIH Collaboratory’s Health Care Systems Interactions Core has published a document entitled Lessons Learned from the NIH Health Care Systems Research Collaboratory Trials. The Principal Investigators of each of the NIH Collaboratory Trials shared their trial-specific experience with the Core to develop the document, which presents problems and solutions for initiation and implementation of pragmatic clinical trials (PCTs). Lessons learned are divided into the following categories: build partnerships, define clinically important questions, assess feasibility, involve stakeholders in study design, consider institutional review board and regulatory issues, and assess potential issues with biostatistics and the analytic plan.
Other tools available from the Health Care Systems Interactions Core include a guidance document entitled Considerations for Training Front-Line Staff and Clinicians on Pragmatic Clinical Trial Procedures and an introduction to PCTs slide set.
Drs. Beverly Green and Gloria Coronado and colleagues have published an article in Clinical Trials describing the challenges of recruiting participants into large, multisite pragmatic clinical trials—particularly at the health system level. STOP CRC is one of the NIH Collaboratory’s pragmatic clinical trial, which are intended to provide a framework of implementation methods and best practices to enable participation of varied health care systems in clinical research.
STOP CRC is testing a culturally tailored, health care system–based program to improve colorectal cancer screening rates in a community-based collaborative network of federally qualified health centers. The authors observed that recruiting sites to participate in pragmatic trials is time-intensive and involves both preparing materials and organizing face-to-face meetings with staff and clinic leaders. Yet little is known about the characteristics of nonparticipating sites and clinic-level factors that may influence willingness to participate in a pragmatic trial.
“Our findings underscore the importance of assessing and reporting recruitment success at the organizational and/or clinic level in order to know the external validity of the findings and may inform future efforts to select and recruit health systems to participate in pragmatic research.” (Coronado, et al. Clin Trials 2015)
