August 23, 2018: PROVEN Team Uses PRECIS-2 to Evaluate Ongoing Trial Implementation

A tool used to rate how the design of a pragmatic clinical trial will influence the real-world applicability of its results can also be used in real time to assess the impact of changes in trial implementation, a recent study published in Trials found.

When designing a pragmatic trial to test the effectiveness of an intervention under “real-world” conditions, researchers use the Pragmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) to assess how a variety of design features may affect the applicability of trial results for the intended users. A pragmatic trial differs from a traditional “explanatory” trial, which tests the efficacy of an intervention under ideal conditions. PRECIS-2 rates design features on a scale from “explanatory” to “pragmatic” within several domains. The end users of trial results, such as health care systems, may need the results to be more or less “pragmatic” on the explanatory–pragmatic spectrum in order to implement the intervention in their own care delivery settings.

In a novel effort, researchers with the Pragmatic Trial of Video Education in Nursing Homes (PROVEN) used PRECIS-2 during the conduct of the trial to assess the effects of midtrial changes in implementation. PROVEN, an ongoing NIH Collaboratory Demonstration Project, is a pragmatic, cluster randomized trial evaluating the effectiveness of video-assisted decision support tools for advance care planning in nursing homes.

In the initial design of PROVEN, design features in the implementation-focused domains of PRECIS-2 were relatively pragmatic, especially in the domain measuring flexibility in the delivery of the intervention. As the trial progressed, adaptations in implementation that were necessary to address challenges in monitoring and protocol adherence led to more explanatory approaches. The investigators concluded that some pragmatic trials, such as those conducted in complex health care systems like nursing homes, “may benefit from a more dynamic approach to implementation which allows for fluidity between pragmatic and explanatory features.” PRECIS-2 can be useful in evaluating the impact of midtrial adaptations with these dynamic approaches to trial implementation.

PROVEN is one of the first large-scale pragmatic trials to be conducted in nursing homes. Learn more about PRECIS-2 in the Living Textbook.

Applying PRECIS Ratings to Collaboratory Pragmatic Trials

A new article published in the journal Trials provides a look at how the  Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS, rating system can be applied to clinical trials designs in order to examine where a given study sits on the spectrum of explanatory versus pragmatic clinical trials.

The PRECIS-2 criteria are used to rate study designs as more or less “pragmatic” according to multiple domains that include participant eligibility, recruitment methods, setting, organization, analysis methods, primary outcomes, and more. In this context, “pragmatic” refers to trials that are designed to study a therapy or intervention in a “real world” setting similar or identical to the one in which the therapy will actually be used. Pragmatic trials stand in contrast to explanatory trials, which are typically designed to demonstrate the safety and efficacy of an intervention under highly controlled conditions and in carefully selected groups of participants, but which may also be difficult to generalize to larger or more varied populations.

Schematic of PRECIS-2 Wheel used to evaluate where a given trial design resides upon the explanatory-pragmatic spectrum.
PRECIS-2 Wheel.  Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.

Clinical trials are almost never wholly “explanatory” or wholly “pragmatic.” Instead, many studies exist somewhere on a spectrum between these two categories. However, understanding how these different attributes apply to trials can help researchers design studies that are optimally fit for purpose, whether that purpose is to describe a biological mechanism (as in an explanatory trial) or to show how effective an intervention is when used across a broad population of patients (as in a pragmatic trial).

In their article in Trials, authors Karin Johnson, Gila Neta, and colleagues  applied PRECIS-2 criteria to 5 pragmatic clinical trials (PCTs) being conducted through the NIH Collaboratory. Each trial was found to rate as “highly pragmatic” across the multiple PRECIS-2 domains, highlighting the tool’s potential usefulness in guiding decisions about study design, but also revealing a number of challenges in applying it and interpreting the results.

Study authors Johnson and Neta will be discussing their findings during the NIH Collaboratory’s Grand Rounds on Friday, January 22, 2016 (an archived version of the presentation will be available the following week).


Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, Stoney CM, Farrell MM, Glasgow RE. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32. doi: 10.1186/s13063-016-1158-y. PMID: 26772801. PMCID: PMC4715340.
You can read more about the NIH Collaboratory PCTs featured as part of this project at the following links:

ABATE (Active Bathing to Eliminate Infection)

LIRE (A pragmatic trial of Lumbar Image Reporting with Epidemiology)

PPACT (Collaborative Care for Chronic Pain in Primary Care)

STOP-CRC (Strategies & Opportunities to Stop Colon Cancer in Priority Populations)

TIME (Time to Reduce Mortality in End-Stage Renal Disease)
Additional Resources

An introductory slide set on PCTs (by study author Karin Johnson) is available from the Living Textbook:

Introduction to Pragmatic Clinical Trials
The University of Colorado Denver - Anschutz Medical Campus publishes an electronic textbook on pragmatic trials:

Pragmatic Trials: A workshop Handbook

 

 

 

Designing Fit-for-Purpose Trials with PRECIS-2


Few clinical trials are entirely explanatory (done in an idealized setting) or entirely pragmatic (done in a usual-care setting); rather, trials are situated somewhere along a continuum of applicability. Pragmatic clinical trials are trials designed with pragmatic qualities and are intended to inform decision makers, including patients, clinicians, administrators, and policymakers, about the relative benefits, burdens, and risks of a health intervention.

To help trialists assess how closely their trial’s design matches its intended purpose, a group of trialists and methodologists developed a design tool, the Pragmatic–Explanatory Continuum Indicator Summary, or PRECIS. Originally implemented in 2008, the wheel-shaped indicator tool recently underwent a revision, leading to PRECIS-2. The revised, validated tool guides trialists to prospectively consider the design of their trial along 9 domains: eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis.

PRECIS-2 Wheel
PRECIS-2 Wheel*

*Kirsty Loudon et al. BMJ 2015;350:bmj.h2147. Copyright 2015 by British Medical Journal Publishing Group. Used by permission.

See the PRECIS-2 introductory video (YouTube) and below links for detailed user information.

Health Informatics Centre website
PRECIS-2 Toolkit
Wheel examples