In an interview at this year’s NIH Pragmatic Trials Collaboratory Steering Committee annual meeting, Drs. Steven George, Vincent Mor, and Angelo Volandes discussed the complexity of intervention delivery in pragmatic clinical trials and the impact it can have on researchers’ ability to discern trial results.

“Without delving deeply into the way in which an intervention can be integrated into an operating system in all of its detail, you will probably make a mistake, and that mistake can impact whether or not your intervention achieves its intended results,” Mor said.

Intervention delivery complexity should be considered early on for pragmatic trials. It is shaped by such factors as new workflows, special training of frontline staff, and the number of components in the intervention.

“We need to understand how we get from point A to point B to point Z, and that’s not something that we do in traditional efficacy trials,” said Volandes.

To characterize this complexity, the NIH Pragmatic Trials Collaboratory worked with its NIH Collaboratory Trial investigators to understand critical drivers of complexity that affected investigators’ ability to implement their interventions and discern treatment effects. The resulting Intervention Delivery Complexity Calculator addresses 6 domains:

• Internal factors pertain primarily to the intervention itself:
  • The degree to which the intervention requires reengineering of existing workflows and tasks
  • The number of components in the intervention
  • The level of familiarity or extra training needed for those delivering the intervention
• External factors are related to intervention delivery at the systems level:
  • The degree to which intervention delivery is dependent on the setting in which it is implemented
  • The number of healthcare systems and clinics involved in delivering the intervention
  • The number of steps between the intervention and the intended outcome

Development of the tool was described in a recent article in Contemporary Clinical Trials.

“We as investigators probably don’t think enough about how health systems operate,” Mor said. “Thinking about intervention delivery complexity can help us start to think about things from an operations context.”

The new tool will be used as part of onboarding trials in the NIH Pragmatic Trials Collaboratory and the National Institute on Aging’s IMPACT Collaboratory, which is focused on pragmatic trials for people living with dementia. The tool can be used during the trial review and funding process all the way through sustainability efforts after a trial has been completed.

George explained, “Intervention delivery complexity is strongly linked to sustainability efforts. Even if you can implement an embedded intervention as part of a trial, if it has a lot of external domain complexity, the intervention could be vulnerable after the trial is completed.”

“By understanding the complexity of intervention delivery, investigators could start thinking about scaled down versions of an intervention, which could help with sustainability,” he added.

The tool was developed to enable conversations with investigators and their teams to think through delivery of the intervention, identify the most complex domains, and consider whether something can be done to reduce complexity.

“The tool moves the idea of complexity regarding delivery of the intervention from something that was an abstract concept to something with structure,” George said.

Future versions of the tool could address the relationship between intervention complexity and adaptations in trials to explore impacts on implementation outcomes. More complex interventions may require a greater number of adaptations to be implemented. Sources of adaptation can include service setting adaptations, target audience adaptations, and mode of delivery adaptations, but there is little understanding about who is making the changes and why.

For more information, see the Intervention Delivery and Complexity chapter of the Living Textbook.