What is a Pragmatic Clinical Trial?
Section 3
Differentiating Between Research and Quality Improvement Activities
Understanding the differences between traditional, “explanatory” randomized controlled trials, pragmatic clinical trials, and quality improvement activities is important, because different ethical and regulatory guidelines may apply. Both randomized controlled trials and pragmatic clinical trials constitute research, defined in federal regulations as an activity intended to create generalizable knowledge (45 CFR 46). Alternatively, quality improvement activities are "data-guided activities designed to bring about immediate improvements in health care delivery in particular settings" (Lynn et al 2007).
When a pragmatic trial is intended to both improve care locally and provide generalizable knowledge, as in a learning health system, the distinction between quality improvement and research is arguably fuzzy (Faden et al 2013), especially as quality improvement techniques become more sophisticated (Finkelstein et al 2015). At its core, however, quality improvement is designed to change local processes to achieve accepted standards of care, whereas pragmatic clinical trials are designed to determine the standards themselves (Finkelstein et al 2015; Nilsen et al 2022). A given clinical trial will have elements that are more pragmatic (for example, a trial accepting all comers with limited exclusion criteria) and elements that are more explanatory (for example, a trial that lacks flexibility in how the intervention is delivered across settings) (Loudon et al 2015).
The Table below summarizes the key differences between randomized controlled trials, pragmatic trials, and quality improvement activities. We explore the continuum of pragmatism in study design in more detail in the next section.
Table. Key Differences Between Traditional Randomized Controlled Trials, Pragmatic Clinical Trials, and Quality Improvement Activities
| Attribute | Randomized Controlled Trial | Pragmatic Clinical Trial | Quality Improvement |
| Who develops the study question? | Researchers | Clinical decision-makers (patients, clinicians, administrators, and policymakers) (Califf and Sugarman 2015) | Clinicians, administrators, and policymakers |
| What is the purpose? | Create generalizable knowledge; determine causes and effects of treatments | Create generalizable knowledge, improve care locally, and inform clinical and policy decisions (Platt et al 2024) | Improve care locally, inform clinical and policy decisions |
| What question does it answer? (Thorpe et al 2009) | Can the intervention work under ideal conditions? | Does this intervention work under usual conditions? (Platt et al 2024) | How do I best implement the intervention? |
| Who is enrolled? | A cohort of patients with strictly defined inclusion and exclusion criteria | Diverse, representative populations (Dember 2022); inclusion and exclusion criteria still apply but tend to be broader | Patients in routine clinical care |
| Who collects data? | Researchers; data collection occurs outside of routine clinical care | Clinicians at the point of care in cooperation with researchers; EHRs and registries are used as sources of research data | Clinicians at the point of care |
| What is studied? | “A biological or mechanistic hypothesis” (Califf and Sugarman 2015) | “The comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level” (Califf and Sugarman 2015) | “Systematic, data-guided activities designed to bring about immediate improvements in health care delivery in particular settings” (Lynn et al 2007) |
| What is compared? | Treatment vs placebo or no treatment | Comparative effectiveness of real-world alternatives | Accepted standards based on published guidelines |
| Is the study randomized to control for potential biases? | Yes, usually at the individual level | Yes; may use experimental designs and randomization schemes such as cluster randomization or stepped-wedge designs (Hemming et al 2015) | Varies |
| What is the setting? | Medical centers designated as research sites | Multiple, heterogeneous settings | Local clinic or hospital; may include multiple clinics or hospitals |
| Adherence to the intervention | Strictly enforced (Zwarenstein et al 2008) | Flexible, as it would be in usual care (Zwarenstein et al 2008) | Normal practice |
| Outcomes | May be surrogates or process measures (Zwarenstein et al 2008) | “Directly relevant to participants, funders, communities, and healthcare practitioners” (Zwarenstein et al 2008) | Directly relevant |
SECTIONS
REFERENCES
Califf RM, Sugarman J. 2015. Exploring the ethical and regulatory issues in pragmatic clinical trials. Clinical Trials. 12:436–441. doi:10.1177/1740774515598334. PMID: 26374676.
Dember LM. 2022. The potential for pragmatic trials to reduce racial and ethnic disparities in kidney disease. J Am Soc Nephrol. 33:1649-1651. doi: 10.1681/ASN.2022030301. PMID: 35922131.
Faden RR, Kass NE, Goodman SN, Pronovost P, Tunis S, Beauchamp TL. 2013. An ethics framework for a learning health care system: A departure from traditional research ethics and clinical ethics. Hastings Cent Rep. Spec No:S16–27. doi:10.1002/hast.134. PMID: 23315888.
Finkelstein JA, Brickman AL, Capron A, et al. 2015. Oversight on the borderline: Quality improvement and pragmatic research. Clin Trials. 12:457–466. doi:10.1177/1740774515597682. PMID: 26374685.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. 2015. The stepped wedge cluster randomised trial: Rationale, design, analysis, and reporting. BMJ. 350:h391–h391. doi:10.1136/bmj.h391. PMID: 25662947.
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. 2015. The PRECIS-2 tool: Designing trials that are fit for purpose. BMJ. 350:h2147. doi:10.1136/bmj.h2147. PMID: 25956159.
Lynn J, Baily MA, Bottrell M, et al. 2007. The ethics of using quality improvement methods in health care. Ann Intern Med. 146:666–673. doi:10.7326/0003-4819-146-9-200705010-00155. PMID: 17438310.
Nilsen P, Thor J, Bender M, Leeman J, Andersson-Gäre B, Sevdalis N. 2022. Bridging the silos: A comparative analysis of implementation science and improvement science. Front Health Serv. 1:817750. doi: 10.3389/frhs.2021.817750. PMID: 36926490.
Platt R, Bosworth HB, Simon GE. 2024. Making pragmatic clinical trials more pragmatic. JAMA. 332:1875-1876. doi: 10.1001/jama.2024.19528. PMID: 39356531.
Thorpe KE, Zwarenstein M, Oxman AD, et al. 2009. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 62:464–475. doi:10.1016/j.jclinepi.2008.12.011. PMID: 19348971.
Zwarenstein M, Treweek S, Gagnier JJ, et al. 2008. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 337:a2390–a2390. doi:10.1136/bmj.a2390. PMID: 19001484.
ACKNOWLEDGMENTS
Contributing editors of previous versions of this chapter include Jonathan McCall, Karen Staman, and Liz Wing of the NIH Pragmatic Trials Collaboratory Coordinating Center.