Speaker
Sean Mann
Senior Policy Analyst
RAND Corporation
Keywords
Development Economics; Spillover; Bias; Pragmatic Clinical Trials
Key Points
- Individually randomized, parallel-group embedded pragmatic trials randomly assign patients to a new healthcare intervention or to a control group that receives some form of usual care. They take place in real-world health settings, where clinical resources can be scarce.
- If an intervention increases patient use of scarce resources – e.g., appointments, hospital beds, provider attention – this can reduce the availability of those resources for the control group and bias results. This is known as negative spillover, or crowding out, in the field of development economics. However, spillover due to resource constraints is absent from the major frameworks that inform clinical trial methods.
- 4 conditions must be met for spillover to affect trial results: 1) Resources are shared across trial arms 2) Care delivery resources are constrained or limited in some way 3) The intervention being studied affects utilization of the scarce resource 4) Having fewer resources available to care for patients affects their health outcomes. These conditions are fairly common in pragmatic trials.
- Spillover is not just a source of bias; it has implications for patient safety. If negative spillover occurs, patients who are assigned to a control arm are no longer receiving usual care as it’s commonly understood. Mann provided strategies that research teams can use to detect spillover or avoid it entirely, including utilizing a cluster-randomized design or a demand-balanced trial design.
- Spillover remains hypothetical, and positive spillover is also possible. Study results affected by spillover can still be a valid indication that an intervention affects care utilization.
- In many pragmatic trials, spillover is likely negligible. Mann and his colleagues at Rand are trying to help determine when spillover concerns should prompt changes to study design or justify additional data collection and analysis. They recently published a paper in Trials that explores the issue in further detail: Mann, S. Negative spillover due to constraints on care delivery: a potential source of bias in pragmatic clinical trials. Trials 25, 833 (2024). https://doi.org/10.1186/s13063-024-08675-9
Discussion Themes
While the Cochrane Risk of Bias Tool v. 2.0 (RoB 2) can be useful for looking at disparities in the level of care provided to the control group versus the intervention group, it exclusively looks at bias due to lack of blinding.
IRB applications could be expanded to include further questions about how resources are being used at a study site, how a study may potentially increase demands on those resources, and how it might affect others who are receiving those health services.
Clinical settings aren’t necessarily a zero sum game; negative spillover will apply in certain contexts, while in others the presence of an intervention will lead to positive spillover or no effect.