In a survey of pragmatic clinical trials across 3 NIH research networks, the complexity of delivering nonpharmacological interventions was similar between pain-related trials and non–pain-related trials. However, pain trials tended to have more intervention components, add more new tasks, and require modifications to existing workflows.
The researchers surveyed study team members from trials in the NIH Pragmatic Trials Collaboratory, the IMPACT Collaboratory, and the Pain Management Collaboratory. All 3 programs support pragmatic clinical trials embedded in healthcare systems, including trials of nonpharmacological interventions for pain.
Though the trials examined in the study had similar intervention complexity, pain trials had slightly greater complexity overall, and the study teams for these trials reported needing to make more adaptations in workflows during the trial to improve the intervention’s fit or effectiveness in real-world settings.
“Change in workflow was an important consideration for intervention delivery for all trials in our study,” wrote lead author Lindsay Ballengee and her coauthors. “Future research should capture detailed, real-time information about the nature of intervention delivery complexity, adaptations, and implementation success to help improve delivery of nonpharmacologic pain interventions,” she wrote. Ballengee is a research fellow with the NIH Pragmatic Trials Collaboratory.
From left: Dr. Steven George, Dr. Vincent Mor, and Dr. Angelo Volandes
In an interview at this year’s NIH Pragmatic Trials Collaboratory Steering Committee annual meeting, Drs. Steven George, Vincent Mor, and Angelo Volandes discussed the complexity of intervention delivery in pragmatic clinical trials and the impact it can have on researchers’ ability to discern trial results.
“Without delving deeply into the way in which an intervention can be integrated into an operating system in all of its detail, you will probably make a mistake, and that mistake can impact whether or not your intervention achieves its intended results,” Mor said.
Intervention delivery complexity should be considered early on for pragmatic trials. It is shaped by such factors as new workflows, special training of frontline staff, and the number of components in the intervention.
“We need to understand how we get from point A to point B to point Z, and that’s not something that we do in traditional efficacy trials,” said Volandes.
To characterize this complexity, the NIH Pragmatic Trials Collaboratory worked with its NIH Collaboratory Trial investigators to understand critical drivers of complexity that affected investigators’ ability to implement their interventions and discern treatment effects. The resulting Intervention Delivery Complexity Calculator addresses 6 domains:
Internal factors pertain primarily to the intervention itself:
The degree to which the intervention requires reengineering of existing workflows and tasks
The number of components in the intervention
The level of familiarity or extra training needed for those delivering the intervention
External factors are related to intervention delivery at the systems level:
The degree to which intervention delivery is dependent on the setting in which it is implemented
The number of healthcare systems and clinics involved in delivering the intervention
The number of steps between the intervention and the intended outcome
“We as investigators probably don’t think enough about how health systems operate,” Mor said. “Thinking about intervention delivery complexity can help us start to think about things from an operations context.”
The new tool will be used as part of onboarding trials in the NIH Pragmatic Trials Collaboratory and the National Institute on Aging’s IMPACT Collaboratory, which is focused on pragmatic trials for people living with dementia. The tool can be used during the trial review and funding process all the way through sustainability efforts after a trial has been completed.
George explained, “Intervention delivery complexity is strongly linked to sustainability efforts. Even if you can implement an embedded intervention as part of a trial, if it has a lot of external domain complexity, the intervention could be vulnerable after the trial is completed.”
“By understanding the complexity of intervention delivery, investigators could start thinking about scaled down versions of an intervention, which could help with sustainability,” he added.
The tool was developed to enable conversations with investigators and their teams to think through delivery of the intervention, identify the most complex domains, and consider whether something can be done to reduce complexity.
“The tool moves the idea of complexity regarding delivery of the intervention from something that was an abstract concept to something with structure,” George said.
Future versions of the tool could address the relationship between intervention complexity and adaptations in trials to explore impacts on implementation outcomes. More complex interventions may require a greater number of adaptations to be implemented. Sources of adaptation can include service setting adaptations, target audience adaptations, and mode of delivery adaptations, but there is little understanding about who is making the changes and why.
In an interview at the NIH Pragmatic Trials Collaboratory Steering Committee’s annual meeting in April, Dr. Steven George discussed how to successfully implement an intervention in a pragmatic clinical trial and how intervention complexity affects implementation across various sites with different local contexts.
“Variation is everywhere, and you have to be willing to ride that wave. It is kind of like being out there waiting to catch a wave into shore, you’re not going to be able to pick how rough that surf is on any given day.”
Successfully implementing and delivering an intervention across multiple organizations is dependent on many factors, including the complexity of the intervention, the stability of the control, and barriers to and promoters of uptake.
Dr. George’s advice is to “Be willing to listen to patients, sites, and providers to help make what you envision for your trial work in a local environment. That flexibility is way to show respect with someone that you are collaborating with. They know their local environment, and if we’re going to ask them as pragmatic trialists to help deliver the intervention, we need to flexible.”
