Skip to content

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
  • Home
  • About
    • NIH Collaboratory
      • Coordinating Center
      • NIH Collaboratory Trials
      • Core Working Groups
      • Steering Committee
      • Distributed Research Network
      • Our Impact
    • Living Textbook
      • Table of Contents
      • How to Use This Site
  • Resources
    • Data and Resource Sharing
    • Training Resources
    • Tools for Researchers
    • Publications
    • Knowledge Repository
  • Webinar
  • Podcast
  • News
    • News Feed
    • Calendar
    • Subscribe
return to home
Subscribe to Newsletter go to twitter feed go to linkedin go to blue sky feed
Search
NIH Collaboratory
Living Textbook of
Pragmatic Clinical Trials

COVID-19 Resources

Access the latest information on COVID-19 for clinical researchers
home button

Rethinking Clinical Trials

A Living Textbook of Pragmatic Clinical Trials

  • Design
    • What is a Pragmatic Clinical Trial?
    • Decentralized Pragmatic Clinical Trials
    • Developing a Compelling Grant Application
    • Experimental Designs and Randomization Schemes
    • Endpoints and Outcomes
    • Analysis Plan
    • Using Electronic Health Record Data
    • Building Partnerships and Teams to Ensure a Successful Trial
    • Intervention Delivery and Complexity
    • Patient Engagement
  • Data, Tools & Conduct
    • Assessing Feasibility
    • Acquiring Real-World Data
    • Assessing Fitness-for-Use of Real-World Data
    • Study Startup
    • Participant Recruitment
    • Monitoring Intervention Fidelity and Adaptations
    • Patient-Reported Outcomes
    • Clinical Decision Support
    • Mobile Health
    • Electronic Health Records–Based Phenotyping
    • Navigating the Unknown
  • Dissemination & Implementation
    • Data Sharing and Embedded Research
    • Dissemination Approaches for Different Audiences
    • Implementation
    • End-of-Trial Decision-Making
  • Ethics & Regulatory
    • Privacy Considerations
    • Identifying Those Engaged in Research
    • Collateral Findings
    • Consent, Disclosure, and Non-Disclosure
    • Data and Safety Monitoring
    • Ethical Considerations of Data Sharing in Pragmatic Clinical Trials
    • Ethics for AI and ML
    • IRB Responsibilities and Procedures

Why Are We Talking About Pragmatic Trials?

CHAPTER SECTIONS

What is a Pragmatic Clinical Trial?


Section 1

Why Are We Talking About Pragmatic Trials?

Expand Contributors

Emily C. O’Brien, PhD
Kevin P. Weinfurt, PhD

Contributing Editor
Damon M. Seils, MA

Healthcare in the United States is complex and expensive. There is a need for more evidence to inform decisions that lead to better, more efficient, more affordable care (Alper and Grossmann 2015). Patients, care providers, researchers, administrators, payers, regulators, and the public agree that the delivery of healthcare should be informed by high-quality scientific evidence regarding the risks and benefits of treatments. Yet, high-quality evidence—generated by randomized controlled trials and disseminated through clinical practice guidelines—is lacking for many therapeutic areas (Tricoci et al 2009; Roos et al 2011; Wright et al 2011; Koh et al 2013; Feuerstein et al 2014; Neuman et al 2014; Fanaroff et al 2019).

When they lack high-quality evidence, clinicians must determine treatment options by making educated guesses based on personal judgment and knowledge of the patient rather than expert consensus (Tricoci et al 2009). Clinicians and patients often simply do not have enough evidence to effectively inform clinical decisions. For example, in cardiology, which arguably has one of the most robust evidence bases among medical specialties, most treatment recommendations are founded upon lower-quality clinical trials, observational studies, or expert opinion (Tricoci et al 2009).

When we survey the US clinical trials enterprise from a broad perspective, we see that the kinds of clinical trials needed to provide high-quality evidence to support treatment decisions are, for the most part, not being conducted. Most clinical trials are too small to provide sufficient statistical power to definitively answer clinical questions, fail to address critical treatment priorities, or have shortcomings in design and execution that limit their usefulness (Califf et al 2012; Pasquali et al 2012; Alexander et al 2013; Goswami et al 2013; Hirsch et al 2013; Lakey et al 2013; Todd et al 2013; Witsell et al 2013; Inrig et al 2014; Subherwal et al 2014). Moreover, the data from many clinical trials are not reported in timely and transparent ways (Anderson et al 2015; Ramachandran et al 2021; DeVito and Goldacre 2021). Adding to these complications, there has been a steady drumbeat of revelations indicating that many findings published in the peer-reviewed literature are unreliable (Ioannidis 2005; Ioannidis 2016; Open Science Collaboration 2015; Le Noury et al 2015).

In modern medicine, clinical research has generally been kept separate from the delivery of routine patient care. As a result, research data are collected using standalone systems. These systems are designed to ensure that the information gathered during research activities is valid and complete. However, maintaining separate systems for research and clinical care comes at a significant cost. In a landmark essay, Ioannidis (2005) sounded an alarm about growing concerns that results obtained from traditional approaches to clinical research may not apply to “real-world” situations. Clinical research is often conducted under artificial conditions with volunteers who may not reflect the complexity of the populations of patients who live with the given disease or condition. Growing awareness of the limitations of traditional approaches to clinical research spurred interest in creating “learning health systems” (Platt et al 2024).

Learning Health Systems and Embedded Clinical Research

In recent years, clinicians, researchers, and healthcare system leaders have advocated for the development of learning health systems, in which tools such as computing power, connectivity, team-based care, and systems engineering techniques produce a culture of continuous learning at lower cost (Institute of Medicine 2013). Approaches to fostering learning health systems continue to evolve, with scholars bringing attention to the importance of health disparities (Parsons et al 2021), trust and shared decision-making (Kelley et al 2015), and real-world data (Butler-Henderson et al 2025), among other issues.

Ideally, clinical trials would be embedded within a system of healthcare delivery where evidence is rapidly and continually fed back into clinical care, and clinical care itself would inform the further development of medical evidence (Embi 2019; Simon 2020). At the same time, the widespread use of electronic health records (EHRs) and advances in information technology and informatics are creating opportunities to combine large, complex datasets (“big data”) in ways that until now were almost unimaginable. As systems for managing data continue to improve in US healthcare systems, the availability of electronic data and advances in artificial intelligence are likewise increasing rapidly.

There is a need for “a different context to clinical research that could speed the discovery and implementation of evidence-based advancements to healthcare delivery. Pragmatic clinical trials (PCTs) are a promising type of trial conducted within real-world health care delivery systems” (Tuzzio and Larson 2019).

Pragmatic clinical trials embedded in healthcare systems represent one approach to building a learning health system to inform real-world practice with digital health data collected at the point of care. Embedded pragmatic trials have the potential to inform policy and practice with high-quality evidence at reduced cost and increased efficiency compared with traditional clinical trials (Platt et al 2024).

The next sections introduce characteristics of embedded pragmatic clinical trials—with examples drawn from the NIH Collaboratory Trials—and point readers to resources available in this Living Textbook that describe best practices for the design, conduct, and dissemination of embedded pragmatic trials.

Next Section

SECTIONS

CHAPTER SECTIONS

sections

  1. Why Are We Talking About Pragmatic Trials?
  2. The Embedded Pragmatic Clinical Trial Ecosystem
  3. Differentiating Between Research and Quality Improvement Activities
  4. Pragmatic Elements: An Introduction to PRECIS-2
  5. PRECIS-2 Case Study
  6. Key Considerations for Pragmatic Trials
  7. Additional Resources

REFERENCES

back to top

Alexander KP, Kong DF, Starr AZ, et al. 2013. Portfolio of clinical research in adult cardiovascular disease as reflected in ClinicalTrials.gov. J Am Heart Assoc. 2:e000009. doi:10.1161/JAHA.113.000009. PMID: 24072529.

Alper J, Grossmann C. 2015. Integrating Research and Practice: Health System Leaders Working toward High-Value Care: Workshop Summary. Washington, DC: National Academies Press.

Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. 2015. Compliance with results reporting at ClinicalTrials.gov. New Engl J Med. 372:1031-1039. doi:10.1056/NEJMsa1409364. PMID: 25760355.

Butler-Henderson K, Arabi S, Wang W. 2025. Data's big three: How learning health systems, artificial intelligence and predictive analytics are transforming healthcare. BMJ Health Care Inform. 32(1):e101414. doi: 10.1136/bmjhci-2024-101414. PMID: 40086805.

Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A. 2012. Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010. JAMA. 307:1838. doi:10.1001/jama.2012.3424. PMID: 22550198.

DeVito NJ, Goldacre B. 2021. Evaluation of compliance with legal requirements under the FDA Amendments Act of 2007 for timely registration of clinical trials, data verification, delayed reporting, and trial document submission. JAMA Intern Med. 181:1128-1130. doi: 10.1001/jamainternmed.2021.2036. PMID: 34028509.

Embi PJ, Richesson R, Tenenbaum J, et al. 2019. Reimagining the research-practice relationship: policy recommendations for informatics-enabled evidence-generation across the US health system. JAMIA Open. 2:2-9. doi:10.1093/jamiaopen/ooy056. PMID: 31984339.

Fanaroff AC, Califf RM, Windecker S, Smith SC Jr, Lopes RD. 2019. Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology guidelines, 2008-2018. JAMA. 321:1069-1080. doi: 10.1001/jama.2019.1122. PMID: 30874755.

Institute of Medicine. 2013. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Smith M, Saunders R, Stuckhardt L, McGinnis JM, eds. Washington, DC: National Academies Press. https://www.ncbi.nlm.nih.gov/books/NBK207225/. Accessed June 23, 2025.

Feuerstein JD, Akbari M, Gifford AE, et al. 2014. Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc. 89:16-24. doi:10.1016/j.mayocp.2013.09.013. PMID: 24388018.

back to top

Goswami ND, Pfeiffer CD, Horton JR, Chiswell K, Tasneem A, Tsalik EL. 2013. The state of infectious diseases clinical trials: a systematic review of ClinicalTrials.gov. PLoS ONE. 8:e77086. doi:10.1371/journal.pone.0077086. PMID: 24146958.

Hirsch BR, Califf RM, Cheng SK, et al. 2013. Characteristics of oncology clinical trials: insights from a systematic analysis of ClinicalTrials.gov. JAMA Intern Med. 173:972. doi:10.1001/jamainternmed.2013.627. PMID: 23699837.

Inrig JK, Califf RM, Tasneem A, et al. 2014. The landscape of clinical trials in nephrology: a systematic review of Clinicaltrials.gov. Am J Kidney Dis. 63:771-780. doi:10.1053/j.ajkd.2013.10.043. PMID: 24315119.

Ioannidis JPA. 2005. Why most published research findings are false. PLoS Med. 2:e124. doi:10.1371/journal.pmed.0020124. PMID: 16060722.

Ioannidis JP. 2016. Why most clinical research is not useful. PLoS Med. 13:e1002049. doi:10.1371/journal.pmed.1002049. PMID: 27328301.

Kelley M, James C, Alessi Kraft S, et al. 2015. Patient perspectives on the learning health system: The importance of trust and shared decision making. Am J Bioeth. 15(9):4-17. doi: 10.1080/15265161.2015.1062163. PMID: 26305741.

Koh C, Zhao X, Samala N, Sakiani S, Liang TJ, Talwalkar JA. 2013. AASLD clinical practice guidelines: a critical review of scientific evidence and evolving recommendations. Hepatology. 58:2142-2152. doi:10.1002/hep.26578. PMID: 23775835.

Lakey WC, Barnard K, Batch BC, Chiswell K, Tasneem A, Green JB. 2013. Are current clinical trials in diabetes addressing important issues in diabetes care? Diabetologia. 56:1226-1235. doi:10.1007/s00125-013-2890-4. PMID: 23564296.

Parsons A, Unaka NI, Stewart C, et al. 2021. Seven practices for pursuing equity through learning health systems: Notes from the field. Learn Health Syst. 5(3):e10279. doi: 10.1002/lrh2.10279. PMID: 34277945.

Platt R, Bosworth HB, Simon GE. 2024. Making pragmatic clinical trials more pragmatic. JAMA. 332:1875-1876. doi: 10.1001/jama.2024.19528. PMID: 39356531.

Ramachandran R, Morten CJ, Ross JS. 2021. Strengthening the FDA's enforcement of ClinicalTrials.gov reporting requirements. JAMA. 326:2131-2132. doi: 10.1001/jama.2021.19773. PMID: 34766971.

ACKNOWLEDGMENTS

back to top

Contributing editors of previous versions of this chapter include Jonathan McCall, Karen Staman, and Liz Wing of the NIH Pragmatic Trials Collaboratory Coordinating Center.

back to top


Version History

Published July 31, 2025

current section :

Why Are We Talking About Pragmatic Trials?

  1. Why Are We Talking About Pragmatic Trials?
  2. The Embedded Pragmatic Clinical Trial Ecosystem
  3. Differentiating Between Research and Quality Improvement Activities
  4. Pragmatic Elements: An Introduction to PRECIS-2
  5. PRECIS-2 Case Study
  6. Key Considerations for Pragmatic Trials
  7. Additional Resources

Citation:

O'Brien EC, Weinfurt KP. What is a Pragmatic Clinical Trial?: Why Are We Talking About Pragmatic Trials?. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Pragmatic Trials Collaboratory. Available at: https://rethinkingclinicaltrials.org/chapters/design/what-is-a-pragmatic-clinical-trial/what-is-a-pragmatic-clinical-trialv2/. Updated September 25, 2025. DOI: 10.28929/267.

Footer Menu

  • How to Use This Site
  • About NIH Collaboratory
  • Enrollment Reporting
  • Grand Rounds
  • Funding Statement
Link to Twitter Link to LinkedIn Link to Blue Sky Link to NIH Collaboratory email

Reference in this Web site to any specific commercial products, process, service, manufacturer, or company does not constitute its endorsement or recommendation by the U.S. Government or National Institutes of Health (NIH). NIH is not responsible for the contents of any “off-site” Web page referenced from this server.

Log in
Privacy Statement
WordPress is a content management system and should not be used to upload any PHI as it is not an environment for which we exercise oversight, meaning you the author are responsible for the content you post. Please use this system accordingly. Site Map