Experimental Designs and Randomization Schemes

Section 1



Elizabeth R. DeLong, PhD

For the NIH Health Care Systems Collaboratory Biostatistics and Study Design Core


Contributing Editor

Jonathan McCall, MS

Pragmatic clinical trials (PCTs) differ from more “traditional” randomized controlled trials (RCTs) largely in terms of their overall purpose. Traditional RCTs can be thought of as explanatory or mechanistic trials that are conducted to evaluate a question regarding a biological mechanism (for example, whether a novel compound is safe and/or efficacious for a given therapeutic purpose). RCTs often take place in idealized conditions that minimize potential confounding and expedite analysis with the goal of ensuring a high degree of internal validity.

PCTs, on the other hand, are often done to evaluate whether a therapy or other intervention is effective in the “real-world” conditions of its proposed use; their ultimate goal is “to improve practice and policy” (Health Care Systems Interactions Core 2015). Califf and Sugarman (2015) propose the following “common sense” definition for a PCT:

[A trial that is] designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level.

Each kind of trial has strengths and weaknesses. Strictly controlled explanatory RCTs seek to maximize internal validity but may not be generalizable outside of controlled settings, because the conditions of study (including the study population) are not representative of more typical patient-treatment settings. They also often require substantial expense and supportive infrastructure and oversight. PCTs, on the other hand, are more likely to maximize external validity and generalizability and may cost less to perform than traditional RCTs. However, less restrictive criteria, a lesser degree of fidelity to intervention, and a lesser degree of monitoring/compliance results in data that are “messier” or less complete, and may also be more subject to changes in the healthcare delivery mechanisms. These considerations need to be addressed through specialized analytical approaches and possibly larger sample sizes.

Importantly, very few trials are entirely explanatory or pragmatic, but exist on a continuum where any given trial contains elements of both in different proportions. The PRECIS-2 system provides one way of thinking about and visualizing explanatory and pragmatic aspects of clinical trial design.

Additional detailed information and checklists for assessing pragmatic trial designs for feasibility can be found in the Assessing Feasibility section of this resource.

Can Traditional RCTs Be Pragmatic?

Many individually randomized trials are optimized for their explanatory power. That is, they are designed to detect differences in the effect of an intervention on particular physiological parameters. The populations enrolled in such trials are typically highly selected in order to exclude conditions that could affect the study endpoints and obscure measurements of treatment differences. As noted in the previous section, while RCTs often have a high degree of internal validity, they may be less generalizable to unselected patient populations where many individuals will have the kinds of comorbid conditions that often result in being excluded from participation in RCTs. For example: it is relatively common for patients with cardiovascular conditions to also suffer from chronic kidney disease, but the latter condition often constitutes an exclusion criterion for many cardiovascular RCTs.

Large Simple Trials

Because “’traditional” RCTs may lack generalizability, are designed to maximize explanatory power, and often require extensive and expensive infrastructure to ensure compliance with complex study protocols, they generally do not conform to the “pragmatic” paradigm. However, although the “pragmatic” terminology is relatively recent, the basic ideas have been implemented for some time under the rubric of “large simple trial” (LST). LSTs typically incorporate extremely simplified protocols that focus on collecting only data that are immediately relevant to the prespecified endpoints; they also typically feature relatively non-restrictive eligibility criteria.

An Older Large Simple Trial and a Contemporary Large Pragmatic Clinical Trial
  • ISIS-2: This 1988 cardiovascular trial randomized more than 17,000 research participants who were admitted to a hospital within 24 hours of experiencing symptoms of acute myocardial infarction to receive treatment with streptokinase, aspirin, both, or neither. Although ISIS-2 was in some respects a “traditional” RCT, it had pragmatic features including minimal eligibility criteria, streamlined data collection, and a “real-world” research setting (ISIS-2 Collaborative Group 1988).
  • ADAPTABLE: This ongoing study is randomizing 20,000 participants who are at elevated risk for heart disease to receive either lower- or higher-dose aspirin in an attempt to ascertain which of these two commonly used doses is better for preventing heart attack and stroke. ADAPTABLE has numerous pragmatic features, including a large sample size drawn from “real-world” populations, data collection centered on using information gathered directly from patients’ electronic health records, and a study endpoint aimed at answering a question that is directly relevant to current clinical practice (ADAPTABLE 2017).



Pragmatic Trials: A Workshop Handbook

This e-book, published by the Colorado Research and Implementation Science Program (CRISP), provides a primer on the design, conduct, and evaluation of PCTs

Large Simple Trials

Findings and products from the Clinical Trials Transformation Initiative’s Large Simple Trials Project


back to top

ADAPTABLE – the Aspirin Study – A Patient-Centered Study. National Patient-Centered Clinical Research Network. Accessed July 26, 2017.

Califf RM, Sugarman J. 2015. Exploring the ethical and regulatory issues in clinical trials. Clin Trials. 12:436-441. doi:0.1177/1740774515598334. PMID: 26374676.

ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. 1988. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 2:349-360. PMID: 2899772.

NIH Collaboratory Health Care Systems Interaction Core. 2015. Introduction to pragmatic clinical trials. Accessed July 26, 2017.


DeLong ER. Experimental Designs and Randomization Schemes: Introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: Updated September 15, 2017.