Evaluating the Trials Using PRECIS-2

For study teams, the broad steps of evaluating a trial using the PRECIS-2 wheel include:

1. Defining the trial’s intention
2. Aligning the design to the intention
3. Identifying the trial’s location on the explanatory-pragmatic continuum (from 1 to 5) for each of the 9 domains (the spokes of the wheel)

Once the intention of the trial is set (pragmatic or explanatory), the designer should align their design choices to their chosen intention across all 9 domains of the PRECIS-2, as outlined in the table below. In this brief discussion, we focus on one of these domains, eligibility, with some mention of the primary outcome.

A very explanatory design approach, with many exclusions and a more physiological outcome (morphology) rather than a patient-centered outcome (mortality), is represented closer to 1 on the PRECIS 2 wheel for each of the domains. Such exclusions may include trial participants who are not adherent to the intervention, who do not respond to treatment, who are at low risk for the primary outcome, and children or older patients. The trial also uses diagnostic tests that are not routine in usual care, to assess both patient eligibility and the primary outcome. On the other hand, a very pragmatic design approach with domains that are set to match as closely as possible those in usual care, that includes almost all patients with the diagnosis, that uses usual diagnostic tests to make the diagnosis, and that chooses outcomes that are directly relevant to patients is represented closer to 5 on the PRECIS-2 wheel.

The Table below summarizes the PRECIS-2 domains and the considerations for each domain.

Table. PRECIS-2 Domains and Considerations

Next, we evaluate the characteristics of both trials and describe the considerations behind each of the 9 PRECIS-2 domains.

Domains: Eligibility, Recruitment, and Setting

In terms of the trial population, Trial 1 (shaded in blue) was restrictive in its inclusion criteria, having individuals aged 16 years and older who started hemodialysis at the renal clinic setting within the last 180 days and who had the capacity to provide informed consent. In addition, individuals were excluded if they could not tolerate cardiac MRI, were pregnant or lactating, or were classified with New York Heart Association class IV heart failure.

In comparison, Trial 2 (shaded in red) had less restrictive inclusion criteria, which widened its applicability to most usual care providers, patient populations, and settings. For efficiency (that is, ease of trial conduct) and to match the nature of policy on dialysate temperature, which is set by each dialysis center, rather than for individual patients, randomization was at the clinic (or cluster) level. In this design, the medical director of each clinic agreed for their clinic to be randomized to either trial arm, and all patients were part of the trial. A waiver of individual patient consent was sought and granted.

Having a highly pragmatic approach for Trial 2 allowed for the rapid enrollment of clinics and patients (both patients receiving chronic dialysis at the time of randomization and future patients starting dialysis). However, because patients at low and high risk for experiencing the primary outcome were included, a large sample size was required, as nonselectivity reduces the effect size.

However, because the Trial 2 protocol became the new clinic policy, there was a need to clearly prescribe and set the dialysate temperature. Also, complex roles and social influences on the process of using a personalized dialysate temperature needed to be assessed.

The flowchart below shows the process of who needed to do what differently, the interrelationships between different roles, and the resulting outcomes. In Trial 2, the leadership at each dialysis clinic changed the local policy to ensure alignment with the assigned temperature protocol. Physicians ordered the assigned temperature protocol for current patients at one time and as new patients received prescriptions for dialysate temperature. Nurses were then trained on the trial protocol and asked to follow the physician's orders to set the dialysate temperature. Nurses were also asked to report any issues or adverse clinical symptoms related to dialysate temperature to the treating physician, who could adjust the dialysate temperature as they deemed appropriate.

Trial 1 (shaded in blue) required specific dialysis machines, research nurses, and specific thermometers to implement the study intervention. In comparison, Trial 2 retained the dialysis machines in use at each included center and allowed the dialysate temperature in the intervention arm to be set in the range of 0.5 to 0.9 degrees below the patient's body temperature. Also, the entire trial—at the clinic level—was implemented by clinicians and nurses rather than researchers. Little local research infrastructure was required in the recruiting clinics to implement the trial due to this flexibility.

In the control arm of Trial 1 (shaded in red), the dialysate temperature was set to 37° C. In Trial 2, clinics were instructed to set the dialysate temperature to at least 36.5° C, unless the patient or their nephrologist preferred a different temperature.

Domains: Flexibility: Adherence and Follow-up

As discussed above, Trial 1 (shaded in blue) employed research staff who closely followed patients for the trial duration. There were regular checkups and feedback sessions between the research staff and participants. In comparison, research staff in Trial 2 (shaded in red) had no contact with participants. However, each month, the clinics were asked to randomly select 15 patients and record the dialysate temperature used for a single session. This allowed the study team to estimate the proportion of patients who were “adhering” to the clinic protocol. It should be noted that the level of adherence for Trial 1 was measured at the participant level, whereas adherence in Trial 2 was measured at the dialysis center. level.

Trial 1 (shaded in blue) had a low PRECIS-2 value (1) for the primary outcome because it tested a change in the resting ejection fraction, which is a surrogate outcome. While important, a change in resting ejection fraction is less relevant to patients. In contrast, for Trial 2, the outcome was cardiovascular-related death or hospital admission for adverse cardiac events, which is of greater relevance to patients and their healthcare providers (PRECIS-2 value 5).

For the analysis domain, Trial 1 used an intention-to-treat approach, which is a highly pragmatic approach. However, nearly 40% of the outcome data was missing due to dropouts or loss to follow-up. From the 73 patients randomized, 37 participants were assigned to the control condition and 36 to the intervention. Of those participants, 28 and 26 were analyzed in the control and intervention arms, respectively. Among those analyzed, data for 5 participants in each arm required multiple imputations for missing outcome data at the 12-month follow-up period.

In contrast, Trial 2 (shaded in red) also used an intention-to-treat approach, but loss to follow-up was expected to be minimal because that typically occurs when study participants leave the (Canadian) province. For example, in Ontario, less than 0.5% the population emigrates from the province each year.