February 22, 2024: Updated Template Provides Guidance for Reporting of Pragmatic Trial Results

An updated template from the NIH Pragmatic Trials Collaboratory provides guidance for the transparent reporting of the primary results of pragmatic clinical trials.

The template includes elements from the Consolidated Standards of Reporting Trials (CONSORT) statement and its extensions. It also addresses secondary use of electronic health record data, involvement of research partners and healthcare systems in the conduct of pragmatic trials, and special ethical and regulatory considerations.

Download the template.

The updated template is organized by the recommended reporting elements presented in the CONSORT checklist and draws on recent experiences and lessons learned from the NIH Collaboratory Trials. Appendices include links to CONSORT and its relevant extensions, the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) tools and resources, and examples of figures to include in pragmatic trial reports.

August 30, 2021: New Living Textbook Materials on Designing a Trial to Match Its Intention

The Living Textbook has recently published materials that explore how randomized trials can be designed to promote both internal and external validity. The new contributions, from Drs. Merrick Zwarenstein, Ahmed Al-Jaishi, and Amit Garg, explain that consideration of the trial’s intention, whether pragmatic or explanatory, is the key to designing a trial that successfully answers its primary research question. While there is a contrast between pragmatic and explanatory intentions, there is not a dichotomy. Instead, trials will vary across the spectrum of design decisions leaning toward choices that match the trial’s purpose. The PRECIS-2 tool can help investigators design their trial to align with its intention. The authors illustrate these points in a new Living Textbook section, PRECIS-2 Case Study, which contrasts the design decisions made for two trials in a renal dialysis setting.

“The purpose should be decided before embarking on designing a trial, and each element of the trial design should be aligned to the chosen purpose.”­– Zwarenstein et al. 2021

Read more at:

November 13, 2020: Pragmatic and Explanatory Attitudes to RCTs: Using the PRECIS-2 Tool to Describe the Design of the MyTEMP Trial (Ahmed Al-Jaishi, PhD; Amit Garg, MD, PhD, Merrick Zwarenstein, MBBCh, MSc, PhD)

Speakers

Ahmed Al-Jaishi, PhD (candidate)
Health Research Methodology
McMaster University

Amit Garg, MD, PhD
Professor, Department of Medicine
Epidemiology & Biostatistics
Western University, London, Canada
Senior Core Scientist, Site Director, and Program Lead
ICES Western Kidney Dialysis & Transplantation Research Program, London, Canada
Nephrologist, London Health Sciences Centre
Victoria Hospital, London, Canada

Merrick Zwarenstein, MBBCh, MSc, PhD
Professor, Department of Family Medicine
Epidemiology & Biostatistics
Western University, London, Canada
Senior Core Scientist
ICES Western Primary Care & Health Systems Research Program, London, Canada

Topic

Pragmatic and Explanatory Attitudes to RCTs: Using the PRECIS-2 Tool to Describe the Design of the MyTEMP Trial

Keywords

Pragmatic trials; PRECIS-2; Canadian Institutes of Health Research (CIHR); Study design; External validity

Key Points

  • Randomized controlled trial (RCT) intentions (attitudes) can be described as: pragmatic intentions provide evidence for decision-makers to choose between interventions, whereas explanatory intentions test a hypothesis about a mechanism.
  • Pragmatic and explanatory trials are not a dichotomy; a good trial design matches the intention of the trial.
  • Trials can be designed with both internal validity and external validity.
  • Different trial intentions, design choices, conclusions, and recommendations lead to different usefulness for decision-making.

Discussion Themes

Can you comment on the retrospective use of PRECIS-2 to dichotomize trials as pragmatic or explanatory?

What suggestions do you have to improve the labeling of trials as explanatory versus pragmatic, since both types are randomized controlled trials (RCTs)?

Read more about the PRECIS-2 tool and the CONSORT statement for pragmatic trials.

Tags

#pctGR, @Collaboratory1

Grand Rounds November 13: Pragmatic and Explanatory Attitudes to RCTs: Using the PRECIS-2 Tool to Describe the Design of the MyTEMP Trial

Speaker:

Ahmed Al-Jaishi, PhD (candidate)
Health Research Methodology
McMaster University

Amit Garg, MD, PhD
Professor, Department of Medicine
Epidemiology & Biostatistics
Western University, London, Canada
Senior Core Scientist, Site Director, and Program Lead
ICES Western Kidney Dialysis & Transplantation Research Program, London, Canada
Nephrologist, London Health Sciences Centre
Victoria Hospital, London, Canada

Merrick Zwarenstein,  MBBCh, MSc, PhD
Professor, Department of Family Medicine
Epidemiology & Biostatistics
Western University, London, Canada
Senior Core Scientist
ICES Western Primary Care & Health Systems Research Program, London, Canada

Topic: Pragmatic and Explanatory Attitudes to RCTs: Using the PRECIS-2 Tool to Describe the Design of the MyTEMP Trial
Date: Friday, November 13, 2020, 1:00-2:00 p.m. ET

Meeting Info: To check whether you have the appropriate players installed for UCF (Universal Communications Format) rich media files, go to https://dukemed.webex.com/dukemed/systemdiagnosis.php.

To join the online meeting:
Go to https://dukemed.webex.com/dukemed/j.php?MTID=m37c037ab1d124abe2b12d39f88a47286

You must log in to the URL first.
Click ‘Audio Conference’
Choose ‘I will call in’, select the Toll number.
Dial in using the information from the dialog box that appears.
Be certain to use the Access Code AND the Attendee ID.

Troubleshooting:
If the URL above does not work, go to dukemed.webex.com and enter:
Meeting Number: 120 486 8317
Meeting Password: 1234

For Audio ONLY:
Call-in toll number (US/Canada): 1-650-479-3207
Access code: 120 486 8317

NOTE: For Toll-free users, the call-back (call me) services are also available.

January 24, 2020: Cardiovascular Trials Over 2 Decades: Progress on Pragmatism? (Justin A. Ezekowitz, MBBCh, MSc)

Speaker

Justin A. Ezekowitz, MBBCh, MSc
Professor, Department of Medicine
Co-Director, Canadian VIGOUR Centre
Director, Cardiovascular Research, University of Alberta
Cardiologist, Mazankowski Alberta Heart Institute

Topic

Cardiovascular Trials Over 2 Decades: Progress on Pragmatism?

Keywords

Pragmatic clinical trials; PRECIS-2; Cardiovascular trials; Enrollment of women

Key Points

  • Pragmatic clinical trials are “designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens and risks of a biomedical or behavioral health intervention at the individual or population level” (Califf & Sugarman, 2015).
  • This study examined how pragmatic or explanatory cardiovascular (CV) randomized controlled trials are; whether the level of pragmatism in CV trials has changed over 2 decades; and whether the proportion of women enrolled in CV trials has changed over 2 decades.
  • No clinical trial is completely explanatory or pragmatic. In this study, trials that scored higher on pragmatism (using the PRECIS-2 tool) had more sites, a larger sample size, longer follow up, and mortality as a primary endpoint.

Discussion Themes

Randomized controlled trials that were published in general medicine journals scored higher in pragmatism than those published in CV journals. Pragmatism has increased over time in CV trials.

While women account for ~45% of the burden of CV diseases, they are underrepresented in CV randomized controlled trials, with less than one-third of trial participants. There was no difference between pragmatic trials and other trials in terms of women’s enrollment.

Initiatives that focus on patient, clinician, and trial design factors are needed to address the gender gap in trial enrollment.

Read more about the PRECIS-2 tool in the Living Textbook, and Dr. Ezekowitz’s research in Trends in the Explanatory or Pragmatic Nature of Cardiovascular Clinical Trials Over 2 Decades (JAMA Cardiology, 2019).

Tags
#pctGR, @Collaboratory1

August 23, 2018: PROVEN Team Uses PRECIS-2 to Evaluate Ongoing Trial Implementation

A tool used to rate how the design of a pragmatic clinical trial will influence the real-world applicability of its results can also be used in real time to assess the impact of changes in trial implementation, a recent study published in Trials found.

When designing a pragmatic trial to test the effectiveness of an intervention under "real-world" conditions, researchers use the Pragmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) to assess how a variety of design features may affect the applicability of trial results for the intended users. A pragmatic trial differs from a traditional "explanatory" trial, which tests the efficacy of an intervention under ideal conditions. PRECIS-2 rates design features on a scale from "explanatory" to "pragmatic" within several domains. The end users of trial results, such as health care systems, may need the results to be more or less "pragmatic" on the explanatory–pragmatic spectrum in order to implement the intervention in their own care delivery settings.

In a novel effort, researchers with the Pragmatic Trial of Video Education in Nursing Homes (PROVEN) used PRECIS-2 during the conduct of the trial to assess the effects of midtrial changes in implementation. PROVEN, an ongoing NIH Collaboratory Trial, is a pragmatic, cluster randomized trial evaluating the effectiveness of video-assisted decision support tools for advance care planning in nursing homes.

In the initial design of PROVEN, design features in the implementation-focused domains of PRECIS-2 were relatively pragmatic, especially in the domain measuring flexibility in the delivery of the intervention. As the trial progressed, adaptations in implementation that were necessary to address challenges in monitoring and protocol adherence led to more explanatory approaches. The investigators concluded that some pragmatic trials, such as those conducted in complex health care systems like nursing homes, "may benefit from a more dynamic approach to implementation which allows for fluidity between pragmatic and explanatory features." PRECIS-2 can be useful in evaluating the impact of midtrial adaptations with these dynamic approaches to trial implementation.

PROVEN is one of the first large-scale pragmatic trials to be conducted in nursing homes. Learn more about PRECIS-2 in the Living Textbook.

Applying PRECIS Ratings to Collaboratory Pragmatic Trials

A new article published in the journal Trials provides a look at how the  Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS, rating system can be applied to clinical trials designs in order to examine where a given study sits on the spectrum of explanatory versus pragmatic clinical trials.

The PRECIS-2 criteria are used to rate study designs as more or less “pragmatic” according to multiple domains that include participant eligibility, recruitment methods, setting, organization, analysis methods, primary outcomes, and more. In this context, “pragmatic” refers to trials that are designed to study a therapy or intervention in a “real world” setting similar or identical to the one in which the therapy will actually be used. Pragmatic trials stand in contrast to explanatory trials, which are typically designed to demonstrate the safety and efficacy of an intervention under highly controlled conditions and in carefully selected groups of participants, but which may also be difficult to generalize to larger or more varied populations.

Schematic of PRECIS-2 Wheel used to evaluate where a given trial design resides upon the explanatory-pragmatic spectrum.
PRECIS-2 Wheel.  Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.

Clinical trials are almost never wholly “explanatory” or wholly “pragmatic.” Instead, many studies exist somewhere on a spectrum between these two categories. However, understanding how these different attributes apply to trials can help researchers design studies that are optimally fit for purpose, whether that purpose is to describe a biological mechanism (as in an explanatory trial) or to show how effective an intervention is when used across a broad population of patients (as in a pragmatic trial).

In their article in Trials, authors Karin Johnson, Gila Neta, and colleagues  applied PRECIS-2 criteria to 5 pragmatic clinical trials (PCTs) being conducted through the NIH Collaboratory. Each trial was found to rate as “highly pragmatic” across the multiple PRECIS-2 domains, highlighting the tool’s potential usefulness in guiding decisions about study design, but also revealing a number of challenges in applying it and interpreting the results.

Study authors Johnson and Neta will be discussing their findings during the NIH Collaboratory’s Grand Rounds on Friday, January 22, 2016 (an archived version of the presentation will be available the following week).


Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, Stoney CM, Farrell MM, Glasgow RE. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32. doi: 10.1186/s13063-016-1158-y. PMID: 26772801. PMCID: PMC4715340.
You can read more about the NIH Collaboratory PCTs featured as part of this project at the following links:

ABATE (Active Bathing to Eliminate Infection)

LIRE (A pragmatic trial of Lumbar Image Reporting with Epidemiology)

PPACT (Collaborative Care for Chronic Pain in Primary Care)

STOP-CRC (Strategies & Opportunities to Stop Colon Cancer in Priority Populations)

TIME (Time to Reduce Mortality in End-Stage Renal Disease)
Additional Resources

An introductory slide set on PCTs (by study author Karin Johnson) is available from the Living Textbook:

Introduction to Pragmatic Clinical Trials
The University of Colorado Denver - Anschutz Medical Campus publishes an electronic textbook on pragmatic trials:

Pragmatic Trials: A workshop Handbook

 

 

 

Designing Fit-for-Purpose Trials with PRECIS-2


Few clinical trials are entirely explanatory (done in an idealized setting) or entirely pragmatic (done in a usual-care setting); rather, trials are situated somewhere along a continuum of applicability. Pragmatic clinical trials are trials designed with pragmatic qualities and are intended to inform decision makers, including patients, clinicians, administrators, and policymakers, about the relative benefits, burdens, and risks of a health intervention.

To help trialists assess how closely their trial’s design matches its intended purpose, a group of trialists and methodologists developed a design tool, the Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS. Originally implemented in 2008, the wheel-shaped indicator tool recently underwent a revision, leading to PRECIS-2. The revised, validated tool guides trialists to prospectively consider the design of their trial along 9 domains: eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.

PRECIS-2 Wheel
PRECIS-2 Wheel*

*Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.

See the PRECIS-2 introductory video (YouTube) and below links for detailed user information.

Health Informatics Centre website
PRECIS-2 Toolkit
Wheel examples