Experimental Designs and Randomization Schemes
Section 1
Introduction
Pragmatic clinical trials (PCTs) differ from traditional randomized controlled trials (RCTs) largely in terms of their purpose. Traditional RCTs can be thought of as explanatory or mechanistic experiments that attempt to minimize potential confounding and ensure a high degree of internal validity. The goal is to determine whether the experimental condition is efficacious when it is the only condition that differs between cases and controls.
PCTs, on the other hand, are often conducted to evaluate whether a therapy or other intervention is effective in the real-world conditions of its proposed use. The ultimate goal of PCTs is to improve practice and policy. Califf and Sugarman (2015) proposed the following definition of a PCT conducted in a healthcare context:
[A trial that is] designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level.
Each kind of trial has strengths and weaknesses. Strictly controlled explanatory RCTs seek to maximize internal validity but may not be generalizable outside controlled settings, because the conditions of the study (including the study population) are not representative of the typical settings in which patients receive care. They also often require substantial expense and supportive infrastructure and oversight. PCTs, on the other hand, are more likely to maximize external validity and generalizability and may cost less to perform than traditional RCTs, especially when they can capitalize on existing data sources such as electronic health records. However, less restrictive criteria, a lesser degree of fidelity to the intervention, and a lesser degree of monitoring or compliance result in data that are "messier" or less complete, and may also be more subject to changes in healthcare delivery mechanisms. These considerations may need to be addressed through specialized analytical approaches and possibly larger sample sizes.
Few clinical trials are entirely explanatory or pragmatic; trials exist on a continuum where any given trial contains elements of both in different proportions. The PRECIS-2 system offers a way of thinking about and visualizing explanatory and pragmatic aspects of clinical trial design.
More information and tools for assessing PCT designs for feasibility can be found in the Assessing Feasibility chapter of the Living Textbook.
Can Traditional RCTs Be Pragmatic?
Many traditional RCTs are optimized for their explanatory power. In other words, they are designed to detect differences in the effects of an intervention on particular prespecified parameters. The populations enrolled in such trials are typically highly selected to exclude conditions that could affect the study endpoints and obscure treatment differences. While RCTs often have a high degree of internal validity, they may be less generalizable to unselected patient populations in which many individuals have comorbid conditions that would lead them to be excluded from participation in the trial. For example, it is relatively common for patients with cardiovascular conditions to also have chronic kidney disease, but the latter condition is an exclusion criterion in many cardiovascular RCTs.
Large Simple Trials
Because traditional RCTs may lack generalizability, are designed to maximize explanatory power, and often require extensive and expensive infrastructure to ensure compliance with complex study protocols, they generally do not conform to the "pragmatic" paradigm. However, although the "pragmatic" terminology is relatively recent, the basic ideas have been implemented for some time in community-based and school-based studies and in clinical research under the rubric of "large simple trials." Large simple trials use simplified protocols that focus on collecting only the data that are immediately relevant to the prespecified endpoints; they also typically feature relatively nonrestrictive eligibility criteria.
An Older Large Simple Trial and a Contemporary Large Pragmatic Clinical Trial
- ISIS-2:This 1988 cardiovascular trial randomized more than 17,000 participants who were admitted to a hospital within 24 hours of experiencing symptoms of acute myocardial infarction to receive treatment with streptokinase, aspirin, both, or neither. Although ISIS-2 was in some respects a traditional RCT, it had pragmatic features, including minimal eligibility criteria, streamlined data collection, and a real-world research setting (ISIS-2 Collaborative Group 1988).
- ADAPTABLE:This study randomly assigned more than 15,000 participants who were at elevated risk for heart disease to receive either lower-dose or higher-dose aspirin in an attempt to ascertain which of these commonly used doses is better for preventing heart attack and stroke. ADAPTABLE had numerous pragmatic features, including a large sample size drawn from real-world populations, data collection centered on using information gathered directly from patients' electronic health records, and a study endpoint aimed at answering a question that is directly relevant to current clinical practice (Jones et al 2021).
SECTIONS
sections
- Introduction
- Statistical Design Considerations
- Cluster Randomized Trials
- Alternative Cluster Randomized Designs
- Stepped-Wedge Designs
- Choosing Between Cluster and Individual Randomization
- Covariate-Constrained Randomization
- Pair Matching and Stratification With Cluster Designs
- Concealment and Blinding
- Designing to Avoid Identification Bias
- Additional Resources
Resources
Why Should I Do A Pragmatic Trial?
Two-minute training module from the NIH Pragmatic Trials Collaboratory's video library
Introduction to Pragmatic Clinical Trials
Presentation from the NIH Pragmatic Trials Collaboratory’s Health Care Systems Interactions Core
Pragmatic Trials: A Workshop Handbook
This e-book from the Colorado Research and Implementation Science Program provides a primer on the design, conduct, and evaluation of PCTs.
Research Methods Resources Website on Group- or Cluster-Randomized Studies
The NIH Office of Extramural Research website provides resources for investigators considering cluster randomized trial designs, including links to NIH webinars, key references, and statements to help investigators prepare sound applications and avoid methodological pitfalls.
Large Simple Trials
Findings and products from the Clinical Trials Transformation Initiative’s Large Simple Trials Project
REFERENCES
Califf RM, Sugarman J. 2015. Exploring the ethical and regulatory issues in clinical trials. Clin Trials. 12:436-441. doi:0.1177/1740774515598334. PMID: 26374676.
ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. 1988. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 2:349-360. PMID: 2899772.
Jones WS, Mulder H, Wruck LM, et al. 2021. Comparative effectiveness of aspirin dosing in cardiovascular disease. N Engl J Med. 384(21):1981-1990. doi: 10.1056/NEJMoa2102137. PMID: 33999548.
current section : Introduction
- Introduction
- Statistical Design Considerations
- Cluster Randomized Trials
- Alternative Cluster Randomized Designs
- Stepped-Wedge Designs
- Choosing Between Cluster and Individual Randomization
- Covariate-Constrained Randomization
- Pair Matching and Stratification With Cluster Designs
- Concealment and Blinding
- Designing to Avoid Identification Bias
- Additional Resources