What is a Pragmatic Clinical Trial?
Section 5
PRECIS-2 Case Study
Focusing on the trial’s intention is the first step in designing a trial that successfully answers its primary research question. For an introduction, read Promoting Both Internal and External Validity: Designing the Trial to Match Its Intention, which describes considerations for choosing a pragmatic or explanatory approach. While both approaches are valuable, their purposes are different and will lead to different design choices. As a result, designs that are more pragmatic will have conclusions and recommendations that are more useful for clinical or policy decision-making, and designs that are more explanatory will be more helpful in expanding scientific knowledge.
Case Study of Trial Design in a Renal Dialysis Setting
This case study illustrates the design of 2 randomized controlled trials that examined the temperature used in hemodialysis. Using the PRECIS-2 framework, we discuss insights about opportunities and constraints that a renal dialysis setting offers and show how the design of the trials aligned with their intention. Trial 1 was more explanatory in intention (Odudu et al. 2015), and Trial 2 was more pragmatic in intention (Al-Jaishi et al. 2020). We begin with some background on the 2 trials and the PRECIS-2 domains that will guide the step-by-step considerations for situating the trials on the pragmatic-explanatory continuum.
Trial 1 Characteristics (blue)
- Personalized dialysis temperature compared with 37°C (98.6°F)
- Unit of randomization: individual patient
- Patients in Nottingham UK enrolled from September 2009 through January 2013
- 1 year follow-up
- 73 patients would have ~11,000 hemodialysis sessions during the trial
- Individual-level consent
- Trial-specific data collection
- Primary outcome: change in the resting ejection fraction measured by cardiac magnetic resonance imaging (MRI) at 12 months compared with baseline
- Cardiac structure, function, and aortic distensibility assessed by cardiac MRI
Trial 2 Characteristics (red)
- Personalized dialysis temperature compared with 36.5°C (97.7°F)
- Unit of randomization: centers in Ontario followed from April 2017 to March 2021
- Maximum 4 years follow-up
- 84 centers, ~13,500 patients (~6000 entered at start of trial, ~7500 entered during trial)
- ~4 million hemodialysis sessions during the trial
- Consent by patient notification via poster and newsletter, allowing opt-out by patients or providers
- Baseline and follow-up data from large databases
- Primary outcome: a composite of cardiac death or hospital admission with myocardial infarction, stroke, or congestive heart failure
Evaluating the Trials Using PRECIS-2
For study teams, the broad steps of evaluating a trial using the PRECIS-2 wheel include:
- Defining the trial’s intention
- Aligning the design to the intention
- Identifying the trial’s location on the explanatory-pragmatic continuum (1-5) for each of the 9 domains (the spokes of the wheel)
In the case study, a very explanatory approach with many exclusions is represented closer to 1. Such exclusions may included expected noncompliers, nonresponders, those at low risk of the primary outcome, children, elderly, or the use of or need for diagnostic tests not used in standard care to assess patient eligibility. On the other hand, a very pragmatic approach with criteria that are essentially identical to those in usual care is represented closer to 5.
PRECIS-2 Domain | Considerations |
(1) Eligibility | To what extent are the participants in the trial like those who would receive this intervention if it was part of usual care? |
(2) Recruitment | How much extra effort is made to recruit participants over and above what would be used in the usual care setting to engage with patients? |
(3) Setting | How different are the settings of the trial from the anticipated usual care setting? |
(4) Organization | How different are the resources, provider expertise and the care delivery organization in the trial and those available in the anticipated usual care situation? Are extra resources added? |
(5) Flexibility: delivery | How different is trial flexibility of delivery from flexibility anticipated in usual care? |
(6) Flexibility: adherence | How different is trial flexibility in monitoring or encouraging adherence from the flexibility anticipated in usual care? |
(7) Follow-up | How intrusive is measurement and follow-up of participants in the trial vs anticipated follow-up in usual care? |
(8) Primary outcome | To what extent is the trial’s primary outcome directly relevant to participants? |
(9) Primary analysis | To what extent are all data included in the analysis of the primary outcome? |
Next, we evaluate the characteristics of both trials and describe the considerations behind each of the 9 PRECIS-2 domains.
(1) Eligibility, (2) Recruitment, (3) Setting
In terms of the trial population, the individual-level trial (blue) was restrictive in its inclusion criteria, having individuals ≥16 years who started hemodialysis at the renal clinic setting within the last 180 days and who had the capacity to provide informed consent. In addition, individuals were excluded if they could not tolerate cardiac magnetic resonance imaging (MRI), were pregnant or lactating, or were classified with New York Heart Association grade IV heart failure.
In comparison, the MyTEMP Trial (red) had less restrictive inclusion criteria in order to increase generalizability. Inclusion was at the clinic (ie, cluster) level, where the medical director agreed for their clinic to be randomized to either trial arm, and whose clinic was expected to provide care per week to at least 15 different adult patients ≥18 years receiving conventional in-clinic hemodialysis.
Having a highly pragmatic approach for MyTEMP allowed for the rapid enrollment of clinics and patients (both patients receiving chronic dialysis at the time of randomization and future patients starting dialysis). However, because patients at low and high risk for experiencing the primary outcome were included, a large sample size was required, as nonselectivity reduces the effect size.
(4) Organization
The individual-level trial scored low (1) on the PRECIS-2 wheel because the intervention required a specific type of thermometer, trained research staff, cardiac MRI, and a dialysis machine with the ability to change the dialysis temperature by steps of 0.1 degrees. In comparison, the MyTEMP trial scored high (5) on the PRECIS-2 wheel because it required no specific expertise or equipment to implement the assigned treatment protocol.
However, because the assigned MyTEMP protocol became the new clinic policy, there was a need to clearly prescribe and set the dialysate temperature. Also, complex roles and social influences on the process of using a personalized dialysate temperature needed to be assessed.
The flowchart below shows the process of who needed to do what differently, the inter-relationships between different roles, and outcomes. In the MyTEMP trial, the leadership at each dialysis clinic changed the local policy to ensure alignment with the assigned temperature protocol. Physicians ordered the assigned temperature protocol for current patients at one time and as new patients received prescriptions for dialysate temperature. Nurses were then trained on the trial protocol and asked to follow physician orders to set the dialysate temperature. Nurses were also asked to report issues or adverse clinical symptoms related to dialysate temperature to the treating physician, who could change the dialysate temperature as they deemed appropriate.
(5) Flexibility: delivery
The individual-level trial required specific dialysis machines, research nurses, and specific thermometers to implement the study intervention. In comparison, the MyTEMP trial allowed the dialysate temperature to be set in the range of 0.5 to 0.9 degrees below the patient's body temperature. Also, the entire trial—at the clinic level—was implemented by clinicians and nurses rather than researchers. This meant that little local research infrastructure was needed in the recruiting clinics to implement the MyTEMP trial with high flexibility.
In the control arm of the individual-level trial, the dialysate temperature of patients was required to be set at 37°C. In the MyTEMP trial, clinics were asked to set the dialysate temperature to at least 36.5°C unless the patient or their nephrologist preferred another temperature be used.
(6) Flexibility: adherence, (7) Follow-up
As discussed above, the individual-level trial employed research staff that closely followed patients for the trial duration. There were regular check-ups and feedback between research staff and participants. In comparison, the MyTEMP trial research staff had no contact with participants. However, each month the clinics were asked to randomly select 15 patients and record the dialysate temperature used for a single session. This allowed the study team to estimate the proportion of patients who were “adhering” to the clinic protocol. It should be noted that the level of adherence for the individual-level trial was measured at the participant level, whereas for the MyTEMP trial, adherence was measured at the clinic level.
(8) Primary outcome, (9) Primary analysis
The individual-level trial had a low PRECIS-2 score (1) for the primary outcome because it tested a change in the resting ejection fraction, which is a surrogate outcome. While important, a change in resting ejection fraction is less relevant to patients. In contrast, for the MyTEMP trial, the outcome was cardiovascular-related death or hospital admission for adverse cardiac events, which is of greater relevance to patients and their healthcare providers (5).
For the analysis domain, the individual-level trial used an intention-to-treat approach, which is a highly pragmatic approach. However, nearly 40% of the outcome data was missing due to dropouts or loss to follow-up. From the 73 patients randomized, 37 participants were assigned to control and 36 to the intervention. Of those participants, 28 and 26 were analyzed in the control and intervention arms, respectively. Among those analyzed, 5 participants in each arm required multiple imputations for missing outcome data at the 12-month follow-up period.
In contrast, the MyTEMP trial will also use an intention-to-treat approach, but loss to follow-up is expected to be minimal because that typically occurs when study participants emigrate from the (Canadian) province. For example, in Ontario less than 0.5% the population per year emigrates from the province.
Summary
Pragmatic trials embedded in routine healthcare delivery are increasingly playing a vital role in filling the large gaps in knowledge about caring for patients on hemodialysis. Several fundamental questions in this healthcare setting appear to be particularly suited to pragmatic approaches to trial design. We used the MyTEMP trial to illustrate a more pragmatic intention, where the authors wanted to test whether use of a clinic-level protocol of personalized temperature-reduced dialysate results in a different rate of cardiovascular-related deaths or hospitalizations than a standard temperature dialysate. The MyTEMP trial was intentionally designed to be highly pragmatic and flexible because of:
- Frequent and predictable patient encounters
- Highly granular and uniform electronic data collection in routine care
- Delivery of care by a small number of provider organizations
There is tremendous interest, both nationally and globally, in increasing the momentum for conducting pragmatic trials. Funders of research, including industry sponsors, are increasingly embracing this approach to reduce costs and generate findings that are rapidly translatable to practice.
SECTIONS
Resources
For an introduction, read Promoting Both Internal and External Validity: Designing the Trial to Match Its Intention.
Additional resources:
For other examples of PCTs conducted in dialysis settings, read about HiLo and TiME.
Review Pragmatic Elements: An Introduction to PRECIS-2, and learn how to evaluate your trial using the PRECIS-2 tool.
Hear more from the authors in the November 13, 2020, Grand Rounds presentation.
REFERENCES
Al-Jaishi AA, McIntyre CW, Sontrop JM, et al. 2020. Major Outcomes With Personalized Dialysate TEMPerature (MyTEMP): Rationale and Design of a Pragmatic, Registry-Based, Cluster Randomized Controlled Trial. Can J Kidney Health Dis. Feb 5. doi:10.1177/2054358119887988. PMID: 32076569.
Odudu A, Eldehni MT, McCann GP, McIntyre CW. 2015. Randomized Controlled Trial of Individualized Dialysate Cooling for Cardiac Protection in Hemodialysis Patients. Clin J Am Soc Nephrol. 10:1408-1417. doi:10.2215/cjn.00200115.