Skip to content

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
  • Home
  • About
    • NIH Collaboratory
      • Coordinating Center
      • NIH Collaboratory Trials
      • Core Working Groups
      • Steering Committee
      • Distributed Research Network
      • Our Impact
    • Living Textbook
      • Table of Contents
      • How to Use This Site
  • Resources
    • Data and Resource Sharing
    • Training Resources
    • Tools for Researchers
    • Publications
    • Knowledge Repository
  • Webinar
  • Podcast
  • News
    • News Feed
    • Calendar
    • Subscribe
return to home
Subscribe to Newsletter go to twitter feed go to linkedin go to blue sky feed
Search
NIH Collaboratory
Living Textbook of
Pragmatic Clinical Trials

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
home button

Rethinking Clinical Trials

A Living Textbook of Pragmatic Clinical Trials

  • Design
    • What is a Pragmatic Clinical Trial?
    • Decentralized Pragmatic Clinical Trials
    • Developing a Compelling Grant Application
    • Experimental Designs and Randomization Schemes
    • Endpoints and Outcomes
    • Analysis Plan
    • Using Electronic Health Record Data
    • Building Partnerships and Teams to Ensure a Successful Trial
    • Intervention Delivery and Complexity
    • Patient Engagement
  • Data, Tools & Conduct
    • Assessing Feasibility
    • Acquiring Real-World Data
    • Assessing Fitness-for-Use of Real-World Data
    • Study Startup
    • Participant Recruitment
    • Monitoring Intervention Fidelity and Adaptations
    • Patient-Reported Outcomes
    • Clinical Decision Support
    • Mobile Health
    • Electronic Health Records–Based Phenotyping
    • Navigating the Unknown
  • Dissemination & Implementation
    • Data Sharing and Embedded Research
    • Dissemination Approaches for Different Audiences
    • Implementation
    • End-of-Trial Decision-Making
  • Ethics & Regulatory
    • Privacy Considerations
    • Identifying Those Engaged in Research
    • Collateral Findings
    • Consent, Disclosure, and Non-Disclosure
    • Data and Safety Monitoring
    • Ethical Considerations of Data Sharing in Pragmatic Clinical Trials
    • Ethics for AI and ML
    • IRB Responsibilities and Procedures

Introduction

CHAPTER SECTIONS

Experimental Designs and Randomization Schemes


Section 1

Introduction

Expand Contributors

Patrick J. Heagerty, PhD
For the NIH Pragmatic Trials Collaboratory Biostatistics and Study Design Core

Contributing Editors

Damon M. Seils, MA
Jonathan McCall, MS

Pragmatic clinical trials (PCTs) differ from traditional randomized controlled trials (RCTs) largely in terms of their purpose. Traditional RCTs can be thought of as explanatory or mechanistic experiments that attempt to minimize potential confounding and ensure a high degree of internal validity. The goal is to determine whether the experimental condition is efficacious when it is the only condition that differs between cases and controls.

PCTs, on the other hand, are often conducted to evaluate whether a therapy or other intervention is effective in the real-world conditions of its proposed use. The ultimate goal of PCTs is to improve practice and policy. Califf and Sugarman (2015) proposed the following definition of a PCT conducted in a healthcare context:

[A trial that is] designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level.

Each kind of trial has strengths and weaknesses. Strictly controlled explanatory RCTs seek to maximize internal validity but may not be generalizable outside controlled settings, because the conditions of the study (including the study population) are not representative of the typical settings in which patients receive care. They also often require substantial expense and supportive infrastructure and oversight. PCTs, on the other hand, are more likely to maximize external validity and generalizability and may cost less to perform than traditional RCTs, especially when they can capitalize on existing data sources such as electronic health records. However, less restrictive criteria, a lesser degree of fidelity to the intervention, and a lesser degree of monitoring or compliance result in data that are "messier" or less complete, and may also be more subject to changes in healthcare delivery mechanisms. These considerations may need to be addressed through specialized analytical approaches and possibly larger sample sizes.

Few clinical trials are entirely explanatory or pragmatic; trials exist on a continuum where any given trial contains elements of both in different proportions. The PRECIS-2 system offers a way of thinking about and visualizing explanatory and pragmatic aspects of clinical trial design.

More information and tools for assessing PCT designs for feasibility can be found in the Assessing Feasibility chapter of the Living Textbook.

Can Traditional RCTs Be Pragmatic?

Many traditional RCTs are optimized for their explanatory power. In other words, they are designed to detect differences in the effects of an intervention on particular prespecified parameters. The populations enrolled in such trials are typically highly selected to exclude conditions that could affect the study endpoints and obscure treatment differences. While RCTs often have a high degree of internal validity, they may be less generalizable to unselected patient populations in which many individuals have comorbid conditions that would lead them to be excluded from participation in the trial. For example, it is relatively common for patients with cardiovascular conditions to also have chronic kidney disease, but the latter condition is an exclusion criterion in many cardiovascular RCTs.

Large Simple Trials

Because traditional RCTs may lack generalizability, are designed to maximize explanatory power, and often require extensive and expensive infrastructure to ensure compliance with complex study protocols, they generally do not conform to the "pragmatic" paradigm. However, although the "pragmatic" terminology is relatively recent, the basic ideas have been implemented for some time in community-based and school-based studies and in clinical research under the rubric of "large simple trials." Large simple trials use simplified protocols that focus on collecting only the data that are immediately relevant to the prespecified endpoints; they also typically feature relatively nonrestrictive eligibility criteria.

An Older Large Simple Trial and a Contemporary Large Pragmatic Clinical Trial

  • ISIS-2:This 1988 cardiovascular trial randomized more than 17,000 participants who were admitted to a hospital within 24 hours of experiencing symptoms of acute myocardial infarction to receive treatment with streptokinase, aspirin, both, or neither. Although ISIS-2 was in some respects a traditional RCT, it had pragmatic features, including minimal eligibility criteria, streamlined data collection, and a real-world research setting (ISIS-2 Collaborative Group 1988).
  • ADAPTABLE:This study randomly assigned more than 15,000 participants who were at elevated risk for heart disease to receive either lower-dose or higher-dose aspirin in an attempt to ascertain which of these commonly used doses is better for preventing heart attack and stroke. ADAPTABLE had numerous pragmatic features, including a large sample size drawn from real-world populations, data collection centered on using information gathered directly from patients' electronic health records, and a study endpoint aimed at answering a question that is directly relevant to current clinical practice (Jones et al 2021).

Next Section

SECTIONS

CHAPTER SECTIONS

sections

  1. Introduction
  2. Statistical Design Considerations
  3. Cluster Randomized Trials
  4. Alternative Cluster Randomized Designs
  5. Stepped-Wedge Designs
  6. Choosing Between Cluster and Individual Randomization
  7. Covariate-Constrained Randomization
  8. Pair Matching and Stratification With Cluster Designs
  9. Concealment and Masking
  10. Designing to Avoid Identification Bias
  11. Additional Resources

Resources

Why Should I Do A Pragmatic Trial?
Two-minute training module from the NIH Pragmatic Trials Collaboratory's video library

Introduction to Pragmatic Clinical Trials
Presentation from the NIH Pragmatic Trials Collaboratory’s Health Care Systems Interactions Core

Pragmatic Trials: A Workshop Handbook
This e-book from the Colorado Research and Implementation Science Program provides a primer on the design, conduct, and evaluation of PCTs.

Research Methods Resources Website on Group- or Cluster-Randomized Studies
The NIH Office of Extramural Research website provides resources for investigators considering cluster randomized trial designs, including links to NIH webinars, key references, and statements to help investigators prepare sound applications and avoid methodological pitfalls.

Large Simple Trials
Findings and products from the Clinical Trials Transformation Initiative’s Large Simple Trials Project

REFERENCES

back to top

Califf RM, Sugarman J. 2015. Exploring the ethical and regulatory issues in clinical trials. Clin Trials. 12:436-441. doi:0.1177/1740774515598334. PMID: 26374676.

ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. 1988. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 2:349-360. PMID: 2899772.

Jones WS, Mulder H, Wruck LM, et al. 2021. Comparative effectiveness of aspirin dosing in cardiovascular disease. N Engl J Med. 384(21):1981-1990. doi: 10.1056/NEJMoa2102137. PMID: 33999548.

back to top


Version History

March 7, 2024: Added a resource to the Resources sidebar (changes made by D. Seils).

Published February 7, 2023

current section :

Introduction

  1. Introduction
  2. Statistical Design Considerations
  3. Cluster Randomized Trials
  4. Alternative Cluster Randomized Designs
  5. Stepped-Wedge Designs
  6. Choosing Between Cluster and Individual Randomization
  7. Covariate-Constrained Randomization
  8. Pair Matching and Stratification With Cluster Designs
  9. Concealment and Masking
  10. Designing to Avoid Identification Bias
  11. Additional Resources

Citation:

Heagerty PJ; for the NIH Pragmatic Trials Collaboratory Biostatistics and Study Design Core. Experimental Designs and Randomization Schemes: Introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Pragmatic Trials Collaboratory. Available at: https://rethinkingclinicaltrials.org/chapters/design/experimental-designs-and-randomization-schemes/experimental-designs-introduction/. Updated March 27, 2024. DOI: 10.28929/202.

Footer Menu

  • How to Use This Site
  • About NIH Collaboratory
  • Enrollment Reporting
  • Grand Rounds
  • Funding Statement
Link to Twitter Link to LinkedIn Link to Blue Sky Link to NIH Collaboratory email

Reference in this Web site to any specific commercial products, process, service, manufacturer, or company does not constitute its endorsement or recommendation by the U.S. Government or National Institutes of Health (NIH). NIH is not responsible for the contents of any “off-site” Web page referenced from this server.

Log in
Privacy Statement
WordPress is a content management system and should not be used to upload any PHI as it is not an environment for which we exercise oversight, meaning you the author are responsible for the content you post. Please use this system accordingly. Site Map