In a new episode of the NIH Pragmatic Trials Collaboratory Podcast, Rich Platt, Hayden Bosworth, and Greg Simon of the NIH Pragmatic Trials Collaboratory discuss their JAMA Viewpoint, “Making Pragmatic Clinical Trials More Pragmatic.”
In the Viewpoint, the authors propose solutions to the discordance between the results of pragmatic trials and the implementation of those results in healthcare settings.
“I think one of the problems with our evidence generating process is that we may think that our
customers are grant review panels or maybe journal editors,” Platt notes in the podcast. “Those may be our short-term customers, but those are not our ultimate customers. Our ultimate customers are people who have to make decisions about healthcare,” he added.
In a JAMA Viewpoint published online this week, leaders from the NIH Pragmatic Trials Collaboratory discuss the discordance between the results of pragmatic clinical trials and the implementation of those results in healthcare settings, even in settings that championed the work.
Coauthors Richard Platt, Hayden Bosworth, and Gregory Simon posit that, to provide evidence that healthcare systems leaders will actually use, changes are necessary:
Trials need to be faster (2 to 3 years)
Trials should consider outcomes that healthcare system leaders care about (such as costs and subgroup analyses)
The evidence required for change should be at the level that healthcare system leaders use (such as not necessarily rejecting the null hypothesis at P < .05 but also considering results of Bayesian methods and interim analyses)
Read the full article.
“We believe it is possible to make pragmatic clinical trials of policies and procedures more useful to delivery systems by accommodating their priorities, introducing flexibility in the level of evidence trials require, shortening the duration of planning and implementation, addressing system leaders’ interest in multiple outcomes and subgroup analyses, encouraging modification of protocols during the implementation phase, and by providing timely interim analyses that can guide decisions about continuing or modifying an intervention,” the authors wrote.
This work was based on an NIH Pragmatic Trials Collaboratory workshop held in 2023, Getting the Right Evidence to Decision-Makers Faster. The workshop explored the critical cycle of evidence generation to decision by healthcare system leaders to implement the findings of pragmatic clinical trials conducted within healthcare systems.
Platt is a cochair of the NIH Collaboratory’s Distributed Research Network and the Harvard Pilgrim Health Care Institute Distinguished Professor of Population Medicine at the Harvard Pilgrim Health Care Institute and Harvard Medical School. Bosworth is a cochair of the Implementation Science Core and a professor in population health sciences at Duke University. Simon is the chair of the Health Care Systems Interactions Core and a senior investigator at the Kaiser Permanente Washington Health Research Institute.
In an interview at the annual NIH Pragmatic Trials Collaboratory Steering Committee meeting, Implementation Science Core cochairs Hayden Bosworth and Devon Check reflected on the Core’s goals and factors for implementation success.
The Implementation Science Core, which launched in 2022, supports implementation-related research aims in pragmatic clinical trials to promote the uptake and sustainability of effective interventions in routine practice.
While the Core is still in its early days, the cochairs have plans for how to best serve the NIH Collaboratory. One of their first initiatives is to develop standardized measurements for evaluating implementation. They are currently gaining consensus around those measurements among the Core’s members.
“We are in the phase of starting to understand the challenges related to implementation during the trial and then afterwards,” Bosworth said.
Stakeholder Engagement Early and Often
Check highlighted early, multilevel stakeholder engagement as a critical step to promoting implementation success.
“The buy-in that you get to conduct a trial can be very different than the buy-in necessary to stand an intervention up in practice,” she said.
Several NIH Collaboratory Trial representatives have highlighted logistics challenges across large health systems as a reoccurring issue.
“Even an intervention that is effective, and potentially also cost-effective, might not be sustained by health systems because it’s just not feasible with the existing resources,” Check said.
It can also be valuable to engage policymakers that can promote implementation.
“Early and ongoing engagement of policymakers who are in the position to make policy changes or requirements based on study findings can sometimes overcome health systems barriers,” she said.
Prioritizing Implementation in Study Design
Bosworth discussed the value of making implementation a priority from the very beginning.
“Are there ways that we can start thinking about implementation from the study design stage?” he said. “Even if it is a terrific study with great findings, if it’s not feasible or too complex to implement, it’s frustrating to wait to consider implementation.”
Engaging with staff can provide insight on the many complexities of multi-level health systems. The study design stage of a pragmatic clinical trial should be seen as an opportunity to promote future implementation, he said.
A Safe Space for Both Failures and Successes
The cochairs highlighted the importance of creating an environment where project teams can share their challenges and failures just as openly as their successes.
“For everything that works, there are probably 10 things that have gone wrong,” Bosworth said. “If we don’t report that, we do the same things over and over. It’s just as valuable to hear what didn’t work so that we can move forward.”
Within the Core, they hope to promote a culture of knowledge sharing as well as collecting and sharing the lessons learned among projects.
“We’re looking forward to consulting, collaborating, and being available as problems and successes arise,” Bosworth said. “And we want to share those lessons learned.”
“Documenting and collecting data on what the challenges are is a useful early step, so that we can characterize the challenges and think about solutions and recommendations for pragmatic trials.” Check said.
Dr. Rosa Gonzalez-Guarda (left) and Dr. Cherise Harrington (right)
The NIH Pragmatic Trials Collaboratory began 2023 with 2 new Core Working Groups focused on health equity and implementation science. The membership of both Cores draws from across the program to support the design and implementation of the NIH Collaboratory Trials.
“By launching these 2 new Core Working Groups, the NIH Pragmatic Trials Collaboratory is reaffirming its commitment to generating knowledge that supports equitable pragmatic research to improve healthcare for all,” said Dr. Adrian Hernandez, co-principal investigator of the NIH Pragmatic Trials Coordinating Center.
The Health Equity Core provides leadership and guidance to help the NIH Collaboratory Trials and investigators be more equitable in research. The Core’s work includes supporting pragmatic trials to address social and structural drivers of inequities, implement patient and community engagement strategies, promote the inclusion and mentorship of historically underrepresented scientists, and develop culturally and contextually aligned research and translation strategies that overcome bias and resonate with patients and communities.
The Health Equity Core is led by Dr. Rosa Gonzalez-Guarda and Dr. Cherise Harrington. Gonzalez-Guarda is an associate professor of nursing at Duke University, faculty lead for the Population Health Research Area of Excellence at Duke’s Center for Nursing Research, and codirector of the Community Engagement Core for Duke Clinical Translational Science Institute. Harrington is a senior researcher and associate professor of public health education at North Carolina Central University. Alex Fist of the Duke Clinical Research Institute serves as the Core’s project manager in the Coordinating Center. The Core’s members include Jessica Lee Barnhill, Sheana Bull, Gaby Castro, Andrea Cheville, Allison Cuthel, Dana Dailey, Juanita Darby, Graham Dore, Julie Fritz, Morgan Fuoco, Christine Goertz, Katharine Lawrence, Vivian Lyons, Keith Marsolo, Alice Pressman, Nina Siman, and Miguel Vazquez.
Dr. Devon Check (left) and Dr. Hayden Bosworth (right)
The Implementation Science Core provides technical support and pragmatic trial expertise for NIH Collaboratory Trials with a specific focus on innovative dissemination and implementation science approaches. The Core will study methods and strategies to promote the uptake of interventions that have proven effective in routine practice, with the aim of improving population health.
The Implementation Science Core is co-led by Dr. Devon Check and Dr. Hayden Bosworth. Check is an assistant professor in population health sciences at Duke University. Bosworth is a professor in population health sciences, psychiatry, and nursing at Duke and deputy director for the Center for Health Services Research in Primary Care at the Durham VA Medical Center. Jill George of the Duke Clinical Research Institute serves as the Core’s project manager in the Coordinating Center. The Core’s members include Kristin Archer, Gaby Castro, Allison Cuthel, Ardith Doorenbos, Carol Greco, Crystal Patil, Sarah Redmond, Isabel Jordan Roth, Stacie Salsbury, Stacy Sterling, Anne Thackeray, Cindy Tofthagen, Katy Trinkley, Miguel Vazquez, and Angelo Volandes.
“Through their work in the areas of implementation science and health equity, the new Cores can help uncover how best to implement improved clinical practices so the benefits reach at-risk and traditionally underserved populations,” Hernandez said.
In a new episode of the NIH Pragmatic Trials Collaboratory Podcast, Rich Platt, Hayden Bosworth, and Greg Simon of the NIH Pragmatic Trials Collaboratory discuss their JAMA Viewpoint, “Making Pragmatic Clinical Trials More Pragmatic.”
In a JAMA Viewpoint published online this week, leaders from the NIH Pragmatic Trials Collaboratory discuss the discordance between the results of pragmatic clinical trials and the implementation of those results in healthcare settings, even in settings that championed the work.
