Among the newest NIH Collaboratory Trials is a study of an intervention to reduce rates of severe maternal morbidity, a health crisis in the United States that disproportionately affects Black patients.
We spoke with principal investigator Stephanie Fitzpatrick about the MOMs Chat & Care Study at the NIH Collaboratory’s 2024 Annual Steering Committee Meeting.
“In the United States, we have the highest rate of maternal morbidity, particularly among Black birthing people,” Fitzpatrick explained. “With findings from our trial, we hope that we can train and support other health systems in being able to provide more intensive wraparound services to get Black birthing people the care they need when they need it to prevent complications and eventually death,” she added.
Fitzpatrick is a professor in the Institute of Health System Science in the Feinstein Institutes for Medical Research at Northwell Health.
The MOMs Chat & Care Study will test the effectiveness of an integrated care model approach at 2 levels of intensity designed to facilitate timely, appropriate care for Black birthing people to reduce their risk for severe maternal morbidity. This NIH Collaboratory Trial is supported by the National Institute of Nursing Research.
In an interview during the NIH Pragmatic Trials Collaboratory’s 2024 Annual Steering Committee Meeting, Tracy Wang of the Patient-Centered Outcomes Research Institute (PCORI) discussed challenges of conducting A vs B pragmatic clinical trials of approved drugs in routine clinical practice. Wang joined Adrian Hernandez, co–principal investigator of the NIH Collaboratory Coordinating Center, and Pearl O’Rourke, cochair of the program’s Ethics and Regulatory Core.
“There are a couple of key challenges, the first of which is, who’s going to pay for the drug, especially if the [drugs being compared] have differential drug costs,” Wang said. “The other aspect is, how do we preserve the rigor of that comparison, [such as] the need for blinding, which is not as pragmatic in routine practice,” she said.
Most pragmatic trials, including the NIH Collaboratory Trials and those conducted through the National Patient-Centered Clinical Research Network (PCORnet), compare standard care with proposed improvements to standard care, or “A vs A-plus trials.” Trials that compare 2 alternative treatments that are in current use, or A vs B trials, are rare.
Wang, a cardiologist and clinical researcher, is PCORI’s chief officer for comparative clinical effectiveness research. Hernandez is a professor of medicine and the vice dean and executive director of the Duke Clinical Research Institute in the Duke University School of Medicine. Prior to her retirement, O’Rourke was the director of human research affairs at Partners HealthCare Systems in Boston and an associate professor of pediatrics at Harvard Medical School.
Three of the newest NIH Collaboratory Trials are supported through the NIH HEAL Initiative℠, or Helping to End Addiction Long-Term Initiative℠, reflecting a special emphasis on developing strategies for the management of chronic pain in rural and remote populations.
“There are many known disparities between urban and rural populations,” said Karen Kehl, a program director at the National Institute of Nursing Research (NINR). “And when we talk about chronic pain, we know that there’s a higher incidence and a higher severity of pain in rural populations, and yet they don’t have access to many of the effective solutions that we have,” Kehl added.
We recently spoke with Kehl, Julie Fritz of the BeatPain Utah trial, and the principal investigators of the AIM-CP, ARBOR-Telehealth, and RAMP trials at the NIH Collaboratory’s 2024 Annual Steering Committee Meeting. They discussed the progress of their studies and the importance of supporting healthcare and promoting health equity in rural communities through pragmatic research.
AIM-CP
AIM-CP will test the implementation of a care management program to address inequities in access to nonpharmacological treatment for chronic pain in rural populations. The principal investigators are Kushang Patel and Sebastian Tong of the University of Washington. The study is supported by NINR. Learn more about AIM-CP.
ARBOR-Telehealth
ARBOR-Telehealth will evaluate the use of a telehealth physical therapy strategy for patients who present to primary care clinics with low back pain in rural communities. The principal investigators are Richard Skolasky and Kevin McLaughlin of Johns Hopkins University. The study is supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases. Learn more about ARBOR-Telehealth.
RAMP
RAMP will evaluate the use of a 12-week mind-body skills training program for rural veterans with pain, including a one-on-one session with a “whole health coach” followed by 11 weekly group sessions to include prerecorded expert-led education videos, mind-body skills training and practice, and group discussions. The principal investigators are Diana Burgess and Roni Evans of the University of Minnesota and Katherine Hadlandsmyth of the University of Iowa. The study is supported by NINR. Learn more about RAMP.
Leaders of the NIH Pragmatic Trials Collaboratory met in Bethesda, Maryland, last month for the program’s 2024 Annual Steering Committee Meeting—an opportunity to network and hold rich discussions on key issues related to pragmatic research.
The program added 9 new trials in the past year. The Annual Steering Committee Meeting provided a collaborative space for new and seasoned NIH Collaboratory Trial investigators to share challenges and lessons learned in conducting their pragmatic trials. They shared methods for increasing patient engagement, overcoming challenges in trial implementation, and creating reusable infrastructure. Seasoned investigators also shared insights into planning for posttrial activities and transitioning to the next trial.
In an interview at the meeting, Coordinating Center co–principal investigators Adrian Hernandez, Lesley Curtis, and Kevin Weinfurt discussed the state of the program and the current pragmatic trials landscape.
“We have more trials working in rural populations,” noted Curtis. “Something that emerged [during the meeting] is the opportunity to bring the Cores together to tackle the issues we’ve been tackling, but with a focus on rural populations, which pose some unique challenges and opportunities,” she said.
The meeting featured a fireside chat, moderated by Wendy Weber, NIH program officer for the NIH Pragmatic Trials Collaboratory, where NIH Director Monica Bertagnolli and FDA Commissioner Robert Califf shared their thoughts on how pragmatic research can help address the nation’s top health priorities. Bertagnolli highlighted an opportunity to bring pragmatic research into a primary care research network that can engage more patients in research. Califf stressed the importance of implementation science and how to maximize the evidence generated from pragmatic trials.
Weber and colleague Beda Jean-Francois of the National Center for Complementary and Integrative Health discussed the future of the NIH Collaboratory program and new directions for pragmatic research.
“I’m always amazed that there is more to learn, new frontiers, new directions—doing trials in rural communities, doing trials to address maternal morbidity,” Weber said as she reflected on key takeaways from the meeting. “We heard a challenge from the NIH Director during this meeting to think about primary care settings and how the NIH Collaboratory can help do those trials and take the lessons we’ve learned into that program,” she said.
From left: Dr. Steven George, Dr. Vincent Mor, and Dr. Angelo Volandes
In an interview at this year’s NIH Pragmatic Trials Collaboratory Steering Committee annual meeting, Drs. Steven George, Vincent Mor, and Angelo Volandes discussed the complexity of intervention delivery in pragmatic clinical trials and the impact it can have on researchers’ ability to discern trial results.
“Without delving deeply into the way in which an intervention can be integrated into an operating system in all of its detail, you will probably make a mistake, and that mistake can impact whether or not your intervention achieves its intended results,” Mor said.
Intervention delivery complexity should be considered early on for pragmatic trials. It is shaped by such factors as new workflows, special training of frontline staff, and the number of components in the intervention.
“We need to understand how we get from point A to point B to point Z, and that’s not something that we do in traditional efficacy trials,” said Volandes.
To characterize this complexity, the NIH Pragmatic Trials Collaboratory worked with its NIH Collaboratory Trial investigators to understand critical drivers of complexity that affected investigators’ ability to implement their interventions and discern treatment effects. The resulting Intervention Delivery Complexity Calculator addresses 6 domains:
Internal factors pertain primarily to the intervention itself:
The degree to which the intervention requires reengineering of existing workflows and tasks
The number of components in the intervention
The level of familiarity or extra training needed for those delivering the intervention
External factors are related to intervention delivery at the systems level:
The degree to which intervention delivery is dependent on the setting in which it is implemented
The number of healthcare systems and clinics involved in delivering the intervention
The number of steps between the intervention and the intended outcome
“We as investigators probably don’t think enough about how health systems operate,” Mor said. “Thinking about intervention delivery complexity can help us start to think about things from an operations context.”
The new tool will be used as part of onboarding trials in the NIH Pragmatic Trials Collaboratory and the National Institute on Aging’s IMPACT Collaboratory, which is focused on pragmatic trials for people living with dementia. The tool can be used during the trial review and funding process all the way through sustainability efforts after a trial has been completed.
George explained, “Intervention delivery complexity is strongly linked to sustainability efforts. Even if you can implement an embedded intervention as part of a trial, if it has a lot of external domain complexity, the intervention could be vulnerable after the trial is completed.”
“By understanding the complexity of intervention delivery, investigators could start thinking about scaled down versions of an intervention, which could help with sustainability,” he added.
The tool was developed to enable conversations with investigators and their teams to think through delivery of the intervention, identify the most complex domains, and consider whether something can be done to reduce complexity.
“The tool moves the idea of complexity regarding delivery of the intervention from something that was an abstract concept to something with structure,” George said.
Future versions of the tool could address the relationship between intervention complexity and adaptations in trials to explore impacts on implementation outcomes. More complex interventions may require a greater number of adaptations to be implemented. Sources of adaptation can include service setting adaptations, target audience adaptations, and mode of delivery adaptations, but there is little understanding about who is making the changes and why.
In an interview at the annual NIH Pragmatic Trials Collaboratory Steering Committee meeting, Implementation Science Core cochairs Hayden Bosworth and Devon Check reflected on the Core’s goals and factors for implementation success.
The Implementation Science Core, which launched in 2022, supports implementation-related research aims in pragmatic clinical trials to promote the uptake and sustainability of effective interventions in routine practice.
While the Core is still in its early days, the cochairs have plans for how to best serve the NIH Collaboratory. One of their first initiatives is to develop standardized measurements for evaluating implementation. They are currently gaining consensus around those measurements among the Core’s members.
“We are in the phase of starting to understand the challenges related to implementation during the trial and then afterwards,” Bosworth said.
Stakeholder Engagement Early and Often
Check highlighted early, multilevel stakeholder engagement as a critical step to promoting implementation success.
“The buy-in that you get to conduct a trial can be very different than the buy-in necessary to stand an intervention up in practice,” she said.
Several NIH Collaboratory Trial representatives have highlighted logistics challenges across large health systems as a reoccurring issue.
“Even an intervention that is effective, and potentially also cost-effective, might not be sustained by health systems because it’s just not feasible with the existing resources,” Check said.
It can also be valuable to engage policymakers that can promote implementation.
“Early and ongoing engagement of policymakers who are in the position to make policy changes or requirements based on study findings can sometimes overcome health systems barriers,” she said.
Prioritizing Implementation in Study Design
Bosworth discussed the value of making implementation a priority from the very beginning.
“Are there ways that we can start thinking about implementation from the study design stage?” he said. “Even if it is a terrific study with great findings, if it’s not feasible or too complex to implement, it’s frustrating to wait to consider implementation.”
Engaging with staff can provide insight on the many complexities of multi-level health systems. The study design stage of a pragmatic clinical trial should be seen as an opportunity to promote future implementation, he said.
A Safe Space for Both Failures and Successes
The cochairs highlighted the importance of creating an environment where project teams can share their challenges and failures just as openly as their successes.
“For everything that works, there are probably 10 things that have gone wrong,” Bosworth said. “If we don’t report that, we do the same things over and over. It’s just as valuable to hear what didn’t work so that we can move forward.”
Within the Core, they hope to promote a culture of knowledge sharing as well as collecting and sharing the lessons learned among projects.
“We’re looking forward to consulting, collaborating, and being available as problems and successes arise,” Bosworth said. “And we want to share those lessons learned.”
“Documenting and collecting data on what the challenges are is a useful early step, so that we can characterize the challenges and think about solutions and recommendations for pragmatic trials.” Check said.
This year’s Annual Steering Committee Meeting for the NIH Pragmatic Trials Collaboratory featured implementation science as one of the topics of focus. In an interview after his keynote presentation, Dr. David Chambers, deputy director for implementation science at the National Cancer Institute, shared his thoughts on opportunities for implementation science in the context of pragmatic trials.
Why Implementation Science Is Important in Pragmatic Research
Chambers described the overlap between implementation science and pragmatic research and how this creates several benefits. Implementation science gives a heightened focus on how to get interventions to be as accessible, well used, and beneficial as possible to populations within the systems and communities in which people are seeking and receiving healthcare.
“The earlier researchers can think about their interventions being used beyond the trial, the better,” Chambers explained. “Implementation science helps to provide a lens for multiple levels of change that may be needed, including supports for patients and families, clinicians, clinics, systems, policymakers, and other key decision-makers.”
Furthermore, implementation science methods can save time in identifying barriers and facilitators for delivering interventions with high quality-knowledge which can be applied toward the ultimate use of interventions.
Chambers encourages researchers to embrace the dynamism that is reflected in our health systems, which is particularly apt for research conducted in the setting of routine care. “It is a given that there will be deviations from the design of an intervention and its implementation, so how can researchers learn from this?” he said.
In terms of sustaining an intervention, a more dynamic approach is needed to how the intervention and the context will change over time, he explained. “Too often we think of sustainment in terms of fixing things in their original state. Sustainment needs to think about how to build in evolution-medicine and our practices are evolving.”
Role of the NIH Pragmatic Trials Collaboratory
According to Chambers, the NIH Pragmatic Trials Collaboratory provides a natural setting for implementation activities because of the close partnerships between the investigative teams and the health systems and community settings where the research occurs. As a result, the program is poised for ongoing learning from the strategies health systems are using to implement a range of different interventions.
There is also an opportunity for pooling lessons across the NIH Collaboratory Trials, such as common measures that can be used to better characterize adaptation, understanding perspectives and needs of patients and clinicians, and approaches related to long-term sustainment or deimplementation of interventions.
Increasingly for NIH Collaboratory trials, issues related to implementation are baked into both the design of interventions and the approach used to test the interventions, which is a testament to the program’s ongoing progress in encouraging the use of implementation science concepts and methods.
For over 20 years, NIH has recognized the gaps in knowledge needed for successful implementation of evidence-based interventions. Across NIH institutes, centers, and offices, they have discussed the common challenges observed when investigators saw a positive result in their trial and were frustrated that it could not be replicated and scaled up in the real world.
NIH funds dissemination and implementation research grants and has a standing review panel, the Science of Implementation in Health and Healthcare, focused on this area where any applicant to NIH can suggest that their grant be reviewed. In addition, NIH supports training opportunities in the field and cohosts an annual scientific conference in partnership with AcademyHealth.
NIH continues to explore the interface between effectiveness and implementation, think about deimplementation, and work on tackling misinformation, all towards advancing how to better apply the evidence generated in research to optimize population health and healthcare.
At the NIH Pragmatic Trials Collaboratory Steering Committee’s annual meeting in May, Dr. Stephanie Morain discussed the results of a recent bioethics initiative to explore ethical considerations for data sharing in the context of pragmatic clinical trials. The project was supported by a supplemental grant award from the NIH through the NIH HEAL Initiative.
Morain and coauthors Juli Bolinger, Kevin Weinfurt, and Jeremy Sugarman published their work in an article in Clinical Trials and a report of stakeholder interviews in Learning Health Systems.
With new data sharing requirements and expectations from funders, journals, and other stakeholders, Morain said the gaps in understanding related to data sharing in pragmatic clinical trials are relevant now more than ever. The supplement was an opportunity to better understand what data sharing policies exist and how well they align with pragmatic trials.
“We suspected that pragmatic clinical trials might raise different challenges compared to sharing data from traditional explanatory trials,” she explained.
Morain said that 2main reasons contributed to why data sharing may be different in the context of pragmatic clinical trials. First, many pragmatic trials use waivers or alterations of informed consent. But much existing ethical guidance for data sharing is based on the argument that sharing data is consistent with participants’ expectations or preferences.
“If we didn’t get explicit permission from participants to participate in the trial, those assumptions [about expectations or preferences] may not hold,” she explained.
The second key difference relates to the fact that pragmatic trials often use existing data from CMS or clinical records. It is often not new data generated for the research purposes, as in many traditional explanatory trials. The embedded nature of pragmatic trial data brings additional security and confidentiality concerns.
The activities of the bioethics supplement were to:
Conduct a systematic literature review to identify the specific features of pragmatic clinical trials that may alter the risk-benefit calculus for data sharing as compared to explanatory trials and other settings with ethically relevant similarities
Through stakeholder interviews, explore data sharing in pragmatic trials as understood by those responsible for the oversight, generation, dissemination, and future use of data from pragmatic trials
Evaluate existing and proposed policies and guidance to promote data sharing
Morain highlighted four key insights from this work.
More Data Needed on Patient Preferences
The first takeaway was that more data are needed on patient preferences in the context of pragmatic clinical trials.
During the qualitative interviews, stakeholders made assumptions based on irrelevant data from traditional explanatory trials. It is clear that many consider patient preferences important, and therefore, more information is needed in the context of pragmatic trials.
“If we don’t actually have permission from the people about who these data relate, we then have to make decisions about either what we think they would prefer or what we think would be in their best interest,” she explained. “Both decisions rely on empirical data to justify them,” she said.
Look Beyond Consent
The second insight was that it may be necessary to look beyond consent.
Many stakeholders in the research focused on consent as the main or only mechanism to demonstrate respect to participants. But Morain said it possible to look beyond informed consent processes to fulfill obligations of respect when sharing data from pragmatic clinical trials, particularly for trials not explicitly asking for consent.
“Consent may be one way that we [researchers] demonstrate respect, but it’s not the only way,” she said.
Healthcare Systems and Institutions Are Key Partners
The third insight was that health systems and individual institutions are key partners. It may be important for them to be transparent that pragmatic research is underway and share findings with patients.
“Institutions are key partners in regards to what data can be shared, and how to do that in a way that maximizes the benefits of sharing but also protects both the patients and the health system,” Morain said.
The Public Can’t Support What It Doesn’t Know Exists
Finally, Morain said that the findings showed that the public can’t support sharing their data if they don’t know this type of research is happening. The medical field does a poor job explaining to patients how their information will be used.
She said that sharing information, even at the aggregate level, can be powerful. Even if informed consent is waived, health systems and institutions can find ways to communicate that pragmatic research is ongoing and highlight its value in advancing science.
“Something we have been advocating for is even if we are not asking for permission, we might still be able to notify,” she said. “Even if researchers don’t have to ask for participant permission, that doesn’t mean participants can’t be told.”
At the NIH Pragmatic Trials Collaboratory Steering Committee Meeting in May, we interviewed Dr. Adrian Hernandez about how to better promote data sharing. Hernandez is a co–principal investigator of the program’s Coordinating Center and the executive director of the Duke Clinical Research Institute.
“We have to change the incentive structure because we still have barriers for sharing data. It’s not being done as often as we need it to be, especially with healthcare system trials,” Hernandez said.
Data sharing is important because it increases transparency, reproducibility, and secondary uses of medical research and is good for society. It also has to potential to advance public health, maximize investment, accelerate learning, and foster collaboration.
One of the first steps to improving data sharing is to increase patient understanding and encourage participation.
People are unaware of what’s possible for reusing data—generating new ideas, tackling different health issues that have not been addressed. People don’t realize that their data can have an expansive use.
The incentives for data sharing are different for researchers, clinicians, and healthcare system leaders.
“For researchers, data sharing efforts could be more readily recognized by linking data sets to [digital object identifiers] in publications. Reuse of data can also be part of recognition for promotion and tenure,” Hernandez said. “For healthcare systems to be more engaged in data sharing, they can promote that they are trying to advance knowledge through research activities and by sharing data.”
The barriers to sharing data include lack of trust and the possibility of misuse.
“People get worried about their data being monetized without their permission, and there are concerns about privacy and being re-identified, and that this somehow will cause harm to people individually,” Hernandez said. He noted that the ethical responsibility to share data generated by publicly funded research must be balanced against the need to protect patient privacy and scientific integrity.
For more information, see the article on data sharing and embedded research in the Annals of Internal Medicine, in which authors from the NIH Collaboratory suggest that data sharing policies must not dissuade healthcare system participation. There is also a Living Textbook chapter on Data Sharing and Embedded Research.
In the days ahead, we will share more interviews with program leaders from the 2023 Steering Committee meeting. All of the materials from the 2023 Steering Committee meeting are now available.
In an interview during the annual Steering Committee meeting, Drs. Rosa Gonzalez-Guarda and Cherise Harrington, cochairs of the Health Equity Core Working Group, shared insights and next steps for health equity in the NIH Pragmatic Trials Collaboratory.
One question discussed at the meeting was how to embed health equity in pragmatic clinical trials. Gonzalez-Guarda said that a key component of addressing health equity is acknowledging the structural and social drivers of that inequity and developing strategies to target them.
“The next phase is to figure out ways that we can encourage embedded pragmatic clinical trials that address the structural causes of health inequities,” said Gonzalez-Guarda, who noted that the National Institutes of Health is supporting more structural interventions.
One way the NIH Pragmatic Trials Collaboratory can lead is by integrating a pragmatic clinical trial lens into structural intervention work, Gonzalez-Guarda said. This work is central to the Health Equity Core, which was launched last fall and began meeting regularly in early 2023.
Health Equity Core members are reviewing a checklist on how to infuse a health equity lens into the research life cycle. As a next step, Harrington and Gonzalez-Guarda would like to ask Core members to think critically about how the checklist can be adapted for pragmatic clinical trials and develop it as a new tool for the research community.
Harrington said another way the Health Equity Core can make an impact is by gathering examples from the NIH Collaboratory Trials of data on underrepresented populations that are not statistically significant but may be clinically significant so that other projects can learn from them.
Drawing from and adding to the NIH Pragmatic Trials Collaboratory’s lessons learned is also a key strategy for the Health Equity Core’s work. Harrington said it is important to share knowledge outside of the research community.
“We need to remember to ask ourselves, 'What am I leaving with this community after the funding is gone that they can still leverage?,’” said Harrington.
Many of the NIH Collaboratory Trials are already implementing health equity plans and engagement strategies to reach diverse patient populations, providers, and community stakeholders.
Gonzalez-Guarda said that it is important to continue this work and to support the careers of junior researchers working on pragmatic clinical trials so they can use engagement practices and be successful. In addition to preparing investigators to engage with diverse communities, part of the engagement work may also be training for communities to better engage with projects in a more meaningful way, Harrington said.
Developing tools and resources is a top priority as the Health Equity Core moves forward, and Gonzalez-Guarda and Harrington look forward to providing guidance for pragmatic clinical trials and continuing the conversation from the Steering Committee meeting.
This year’s 
According to Chambers, the NIH Pragmatic Trials Collaboratory provides a natural setting for implementation activities because of the close partnerships between the investigative teams and the health systems and community settings where the research occurs. As a result, the program is poised for ongoing learning from the strategies health systems are using to implement a range of different interventions.
