Experimental Designs and Randomization Schemes

Section 1



Elizabeth R. DeLong, PhD

For the NIH Health Care Systems Collaboratory Biostatistics and Study Design Core


Contributing Editors

Jonathan McCall, MS

Damon M. Seils, MA

Pragmatic clinical trials (PCTs) differ from “traditional” randomized controlled trials (RCTs) largely in terms of their overall purpose. Traditional RCTs can be thought of as explanatory or mechanistic experiments that attempt to minimize potential confounding and ensure a high degree of internal validity. The goal is to determine whether the experimental condition is efficacious when it is the only condition that differs between cases and controls.

PCTs, on the other hand, are often conducted to evaluate whether a therapy or other intervention is effective in the “real-world” conditions of its proposed use; their ultimate goal is “to improve practice and policy” (Health Care Systems Interactions Core 2015). Califf and Sugarman (2015) propose the following “common sense” definition for a PCT when conducted in a healthcare context:

[A trial that is] designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level.

Each kind of trial has strengths and weaknesses. Strictly controlled explanatory RCTs seek to maximize internal validity but may not be generalizable outside controlled settings, because the conditions of study (including the study population) are not representative of more typical patient-treatment settings. They also often require substantial expense and supportive infrastructure and oversight. PCTs, on the other hand, are more likely to maximize external validity and generalizability and may cost less to perform than traditional RCTs, especially when they can capitalize on existing data sources such as the electronic health record (EHR). However, less restrictive criteria, a lesser degree of fidelity to intervention, and a lesser degree of monitoring/compliance results in data that are “messier” or less complete, and may also be more subject to changes in the healthcare delivery mechanisms. These considerations may need to be addressed through specialized analytical approaches and possibly larger sample sizes.

Importantly, very few trials are entirely explanatory or pragmatic, but exist on a continuum where any given trial contains elements of both in different proportions. The PRECIS-2 system provides one way of thinking about and visualizing explanatory and pragmatic aspects of clinical trial design.

Additional detailed information and checklists for assessing pragmatic trial designs for feasibility can be found in the Assessing Feasibility section of this resource.

Can Traditional RCTs Be Pragmatic?

Many traditional randomized trials are optimized for their explanatory power. That is, they are designed to detect differences in the effect of an intervention on particular prespecified parameters. The populations enrolled in such trials are typically highly selected in order to exclude conditions that could affect the study endpoints and obscure measurements of treatment differences. As noted in the previous section, while RCTs often have a high degree of internal validity, they may be less generalizable to unselected patient populations where many individuals will have the kinds of comorbid conditions that often result in being excluded from participation in RCTs. For example: it is relatively common for patients with cardiovascular conditions to also suffer from chronic kidney disease, but the latter condition often constitutes an exclusion criterion for many cardiovascular RCTs.

Large Simple Trials

Because “’traditional” RCTs may lack generalizability, are designed to maximize explanatory power, and often require extensive and expensive infrastructure to ensure compliance with complex study protocols, they generally do not conform to the “pragmatic” paradigm. However, although the “pragmatic” terminology is relatively recent, the basic ideas have been implemented for some time in community- and school-based studies and also under the rubric of the “large simple trial” (LST). LSTs typically incorporate extremely simplified protocols that focus on collecting only data that are immediately relevant to the prespecified endpoints; they also typically feature relatively non-restrictive eligibility criteria.

An Older Large Simple Trial and a Contemporary Large Pragmatic Clinical Trial

  • ISIS-2: This 1988 cardiovascular trial randomized more than 17,000 research participants who were admitted to a hospital within 24 hours of experiencing symptoms of acute myocardial infarction to receive treatment with streptokinase, aspirin, both, or neither. Although ISIS-2 was in some respects a “traditional” RCT, it had pragmatic features including minimal eligibility criteria, streamlined data collection, and a “real-world” research setting (ISIS-2 Collaborative Group 1988).
  • ADAPTABLE: This ongoing study is randomizing 20,000 participants who are at elevated risk for heart disease to receive either lower- or higher-dose aspirin in an attempt to ascertain which of these two commonly used doses is better for preventing heart attack and stroke. ADAPTABLE has numerous pragmatic features, including a large sample size drawn from “real-world” populations, data collection centered on using information gathered directly from patients’ electronic health records, and a study endpoint aimed at answering a question that is directly relevant to current clinical practice (ADAPTABLE 2017).




Pragmatic Trials: A Workshop Handbook

This e-book, published by the Colorado Research and Implementation Science Program (CRISP), provides a primer on the design, conduct, and evaluation of PCTs

Research Methods Resources Website on Group- or Cluster-Randomized Studies

The National Institutes of Health (NIH) Office of Extramural Research website provides resources for investigators considering these group- or cluster-randomized designs, including lists of NIH webinars, key references, and statements to help investigators prepare sound applications and avoid methodological pitfalls.

Large Simple Trials

Findings and products from the Clinical Trials Transformation Initiative’s Large Simple Trials Project


back to top

ADAPTABLE – the Aspirin Study – A Patient-Centered Study. National Patient-Centered Clinical Research Network. Accessed July 26, 2017.

Califf RM, Sugarman J. 2015. Exploring the ethical and regulatory issues in clinical trials. Clin Trials. 12:436-441. doi:0.1177/1740774515598334. PMID: 26374676.

ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. 1988. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 2:349-360. PMID: 2899772.

NIH Collaboratory Health Care Systems Interaction Core. 2015. Introduction to pragmatic clinical trials. Accessed July 26, 2017.

Version History

January 15, 2019: Updated text describing differences between RCTs and PCTs and the history of pragmatic approaches in other settings as part of annual content update (changes made by D. Seils).

Published August 25, 2017


DeLong ER. Experimental Designs and Randomization Schemes: Introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: Updated January 15, 2019. DOI: 10.28929/002.