In a survey of pragmatic clinical trials across 3 NIH research networks, the complexity of delivering nonpharmacological interventions was similar between pain-related trials and non–pain-related trials. However, pain trials tended to have more intervention components, add more new tasks, and require modifications to existing workflows.
The researchers surveyed study team members from trials in the NIH Pragmatic Trials Collaboratory, the IMPACT Collaboratory, and the Pain Management Collaboratory. All 3 programs support pragmatic clinical trials embedded in healthcare systems, including trials of nonpharmacological interventions for pain.
Though the trials examined in the study had similar intervention complexity, pain trials had slightly greater complexity overall, and the study teams for these trials reported needing to make more adaptations in workflows during the trial to improve the intervention’s fit or effectiveness in real-world settings.
“Change in workflow was an important consideration for intervention delivery for all trials in our study,” wrote lead author Lindsay Ballengee and her coauthors. “Future research should capture detailed, real-time information about the nature of intervention delivery complexity, adaptations, and implementation success to help improve delivery of nonpharmacologic pain interventions,” she wrote. Ballengee is a research fellow with the NIH Pragmatic Trials Collaboratory.
During the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting in May, Greg Simon and Steve George sat down to discuss challenges and lessons learned from NIH Collaboratory Trials on healthcare system and patient engagement.
Simon said it is key for embedded pragmatic clinical trials to build long-term relationships with healthcare systems, starting with finding out what are the needs of the healthcare system, the clinicians, and the patients they care for, and then building trials that address the questions they care about.
But healthcare systems change rapidly, which can lead to challenges, such as turnover at different levels of leadership, including the top level of the healthcare system, the clinic management level, and the provider level.
“It is important for clinical trial investigators to think about how do I form relationships with those people who are likely to be here for a while? How do I form relationships at multiple levels with different people,” Simon said.
With patient engagement, George said the challenges have evolved with the NIH Collaboratory Trials. When the NIH Collaboratory started, the initial challenge was whether to do patient engagement or not. Today, patient engagement is more of an expectation for trials, so the questions have become more nuanced, George said. Now trials must consider at what phase of the trial are you going to do patient engagement; who are you going to involve; and are you going to do it throughout the trial?
The NIH Collaboratory’s Living Textbook includes a new chapter on Patient Engagement to help investigators think through these questions.
“One of the most effective resources is the reporting and the information sharing across trials. There are a lot of models of how this has been done. There are a lot of models of how I would do it differently if I did it again, and I think that open sharing is very powerful for people to see because people like examples,” George said.
Simon, a senior investigator at Kaiser Permanente Washington Health Research Institute, is the chair of the NIH Collaboratory’s Health Care Systems Interactions Core and a member of the Coordinating Center leadership team. George is the Laszlo Ormandy Distinguished Professor of Orthopaedic Surgery at Duke University.
From left: Dr. Steven George, Dr. Vincent Mor, and Dr. Angelo Volandes
In an interview at this year’s NIH Pragmatic Trials Collaboratory Steering Committee annual meeting, Drs. Steven George, Vincent Mor, and Angelo Volandes discussed the complexity of intervention delivery in pragmatic clinical trials and the impact it can have on researchers’ ability to discern trial results.
“Without delving deeply into the way in which an intervention can be integrated into an operating system in all of its detail, you will probably make a mistake, and that mistake can impact whether or not your intervention achieves its intended results,” Mor said.
Intervention delivery complexity should be considered early on for pragmatic trials. It is shaped by such factors as new workflows, special training of frontline staff, and the number of components in the intervention.
“We need to understand how we get from point A to point B to point Z, and that’s not something that we do in traditional efficacy trials,” said Volandes.
To characterize this complexity, the NIH Pragmatic Trials Collaboratory worked with its NIH Collaboratory Trial investigators to understand critical drivers of complexity that affected investigators’ ability to implement their interventions and discern treatment effects. The resulting Intervention Delivery Complexity Calculator addresses 6 domains:
Internal factors pertain primarily to the intervention itself:
The degree to which the intervention requires reengineering of existing workflows and tasks
The number of components in the intervention
The level of familiarity or extra training needed for those delivering the intervention
External factors are related to intervention delivery at the systems level:
The degree to which intervention delivery is dependent on the setting in which it is implemented
The number of healthcare systems and clinics involved in delivering the intervention
The number of steps between the intervention and the intended outcome
“We as investigators probably don’t think enough about how health systems operate,” Mor said. “Thinking about intervention delivery complexity can help us start to think about things from an operations context.”
The new tool will be used as part of onboarding trials in the NIH Pragmatic Trials Collaboratory and the National Institute on Aging’s IMPACT Collaboratory, which is focused on pragmatic trials for people living with dementia. The tool can be used during the trial review and funding process all the way through sustainability efforts after a trial has been completed.
George explained, “Intervention delivery complexity is strongly linked to sustainability efforts. Even if you can implement an embedded intervention as part of a trial, if it has a lot of external domain complexity, the intervention could be vulnerable after the trial is completed.”
“By understanding the complexity of intervention delivery, investigators could start thinking about scaled down versions of an intervention, which could help with sustainability,” he added.
The tool was developed to enable conversations with investigators and their teams to think through delivery of the intervention, identify the most complex domains, and consider whether something can be done to reduce complexity.
“The tool moves the idea of complexity regarding delivery of the intervention from something that was an abstract concept to something with structure,” George said.
Future versions of the tool could address the relationship between intervention complexity and adaptations in trials to explore impacts on implementation outcomes. More complex interventions may require a greater number of adaptations to be implemented. Sources of adaptation can include service setting adaptations, target audience adaptations, and mode of delivery adaptations, but there is little understanding about who is making the changes and why.
