In this episode of the NIH Collaboratory podcast, Drs. Greg Simon and Lorella Palazzo discuss their recent Living Textbook Chapter, “End-of-Trial Decision-Making.”
Read the transcript. For alerts about new episodes, subscribe for free on Spotify, Amazon Music, Apple Podcasts, or SoundCloud.
In an article published online ahead of print, leaders from the NIH Pragmatic Trials Collaboratory share lessons learned about the importance of independent oversight by a safety office or data and safety monitoring board in pragmatic clinical trials, even for trials deemed to have minimal risk.
Challenges specific to pragmatic trials include:
“We recommend regular, rigorous data quality checks, ongoing monitoring of adherence to interventions, and including someone who is knowledgeable about pragmatic clinical trials and novel research designs in the development of Data and Safety Monitoring Plans and Data and Safety Monitoring Boards,” the authors wrote.
By attempting to reflect real-world conditions, pragmatic trials are conducted in settings that cannot be closely controlled. Therefore, close monitoring is critical for a successful study that produces meaningful results, whether it be by independent monitors or data and safety monitoring boards.
The authors drew on experiences from 7 of the NIH Collaboratory Trials and the expertise of the Coordinating Center, the Ethics and Regulatory Core, the Biostatistics and Study Design Core, and the Health Care Systems Interactions Core.
The article was published in Contemporary Clinical Trials.
During the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting in May, Greg Simon and Steve George sat down to discuss challenges and lessons learned from NIH Collaboratory Trials on healthcare system and patient engagement.
Simon said it is key for embedded pragmatic clinical trials to build long-term relationships with healthcare systems, starting with finding out what are the needs of the healthcare system, the clinicians, and the patients they care for, and then building trials that address the questions they care about.
But healthcare systems change rapidly, which can lead to challenges, such as turnover at different levels of leadership, including the top level of the healthcare system, the clinic management level, and the provider level.
“It is important for clinical trial investigators to think about how do I form relationships with those people who are likely to be here for a while? How do I form relationships at multiple levels with different people,” Simon said.
With patient engagement, George said the challenges have evolved with the NIH Collaboratory Trials. When the NIH Collaboratory started, the initial challenge was whether to do patient engagement or not. Today, patient engagement is more of an expectation for trials, so the questions have become more nuanced, George said. Now trials must consider at what phase of the trial are you going to do patient engagement; who are you going to involve; and are you going to do it throughout the trial?
The NIH Collaboratory’s Living Textbook includes a new chapter on Patient Engagement to help investigators think through these questions.
“One of the most effective resources is the reporting and the information sharing across trials. There are a lot of models of how this has been done. There are a lot of models of how I would do it differently if I did it again, and I think that open sharing is very powerful for people to see because people like examples,” George said.
Simon, a senior investigator at Kaiser Permanente Washington Health Research Institute, is the chair of the NIH Collaboratory’s Health Care Systems Interactions Core and a member of the Coordinating Center leadership team. George is the Laszlo Ormandy Distinguished Professor of Orthopaedic Surgery at Duke University.
The NIH Pragmatic Trials Collaboratory Heath Care Systems Interactions Core supports and facilitates productive collaboration between researchers, clinicians, and health system leaders to conduct effective, relevant embedded pragmatic clinical research.
Health Care Systems Interactions Core Co-Chairs Dr. Eric B. Larson and Dr. Gregory Simon discussed the Core’s progress over the last 10 years in an interview at the NIH Pragmatic Trials Collaboratory Steering Committee meeting in April.
Over the last 10 years, the Health Care Systems Interactions (HCS) Core has allowed researchers to learn about working with healthcare delivery systems. Knowledge that is now common, was unknown when the Core was started, such as how dynamic healthcare delivery systems are and how the capabilities of and changes to the electronic health record can impact pragmatic clinical trials.
Through this type of discovery, the HCS Core has helped researchers become more sensitive to and aware of the priorities of healthcare delivery systems, resulting in better collaboration.
“The researchers’ priorities are usually not the same as the priorities of the people we are working with, whether they are patients, providers, or delivery systems. You have to know what other people’s purpose and drivers are and find a way to adapt,” said Larson. “We have learned and taken pretty seriously this idea of a learning health system with bidirectional engagement from research and from elements of the delivery system.”
The work of the HCS Core and NIH Pragmatic Trials Collaboratory has created a safe haven where researchers can share experiences and advance the field with common learning.
Simon sees the HCS Core as having internal and external missions. The internal mission is to support NIH Collaboratory Trials and be a community where researchers can come together, share their trials and tribulations, and experts in the Core can help these projects be successful, he described.
The external mission is focused on generalizable knowledge and advocacy. The HCS Core has shared knowledge with the research community and funders through publications and meetings and is advocating for research that includes the healthcare delivery system perspective.
“The discussion we are having is not just how do I work with healthcare systems to do my research, but how do I engage with healthcare systems about what research we should be doing, what are the right questions we should be asking for the studies that will be happening 5 years from now not the studies that are already underway,” Simon said.
Another lesson the HCS Core has learned is the importance of being flexible and adjusting. This lesson has been particularly relevant during the COVID-19 pandemic.
“COVID-19 is an extreme case of health system overwhelm, but I think we need to recognize that if we are going to serve people that have been traditionally not been well-served by the healthcare system, we will often be dealing with health systems that are chronically overwhelmed,” said Simon. “How do we do research in those settings? There are some really interesting challenges to think about.”
The HCS Core is focused on continued engagement between researchers and healthcare delivery systems that results in implementable new knowledge.
“My belief is that if we have the upstream involvement and are engaged in research projects that matter to the delivery system from the patients all the way up to the executives, we have a much better chance that when a result is valuable it becomes implementable and spreads to benefit everybody,” Larson said.
See the complete materials from the 2022 Steering Committee meeting.
Leaders of the NIH Collaboratory’s Health Care Systems Interactions Core Working Group spoke in a recent interview about the impacts of the COVID-19 pandemic on the NIH Collaboratory Trials.
“Some of the projects are facing healthcare systems that are on pause for research,” said Leah Tuzzio, a senior research associate at Kaiser Permanente Washington Health Research Institute (KPWHRI) and a member of the Core. “It’s happened before when policy changes or when [electronic health record] systems change or when someone important leaves, but the pandemic has had a huge impact,” she said.
“Healthcare systems are strongly influenced by the environment and by changes in their environment, and so any pragmatic trial that’s really embedded in the healthcare system will be affected by the environment around it,” said Dr. Eric Larson, a KPWHRI senior investigator and the chair of the Core. Healthcare systems participating in the NIH Collaboratory NIH Collaboratory Trials have been overwhelmed by their number one priority, which—in addition to caring for their patients—is adapting to COVID, Larson said.
In addition to guiding the NIH Collaboratory Trials through pandemic-related challenges, the Health Care Systems Interactions Core has been working on several long-term projects.
“One of the things that we’re currently working on is a typology of the healthcare systems that have participated in [the GRACE and BeatPain Utah NIH Collaboratory Trials] as well as the projects that came before,” said Core project manager Rachel Hays. The Core is surveying NIH Collaboratory Trial investigators about what lessons they would pass on to future pragmatic trial investigators about building partnerships with their participating healthcare systems, she said.
The Health Care Systems (HCS) Interactions Core of the NIH Collaboratory has created a new guidance document: Resources for Frequently Asked Questions. The document compiles common questions from researchers and points to resources that provide potential solutions, including Living Textbook chapters and journal articles. The topics include evaluation strategies, anticipating and adapting to changes, dissemination and implementation, and stakeholder engagement.
The NIH Collaboratory Core Working Groups have created a series of handouts to help investigators conduct pragmatic trials:
