August 14, 2023: Program Leaders Reflect on Cross-Core Collaboration and Opportunities for the Future

Headshots of Dr. Kevin Weinfurt and Dr. Greg Simon
From left: Kevin Weinfurt and Greg Simon

In an interview during the program’s Annual Steering Committee Meeting, Drs. Kevin Weinfurt and Greg Simon reflected on cross-Core collaboration within the NIH Pragmatic Trials Collaboratory and opportunities for growth in the future.

More than a decade ago, when the NIH Pragmatic Trials Collaboratory got its start, the focus was on identifying and answering the big questions in terms of how to do research embedded in routine healthcare, said Weinfurt, co–principal investigator of the NIH Collaboratory’s Coordinating Center. As a result, during the founding years, the NIH Collaboratory’s Core Working Groups primarily worked on addressing the big questions that were specific to each Core (eg, ethics and regulatory issues, electronic health record challenges).

After working through many of the early questions, the program’s approach has evolved in recent years.

“As we tackled some of the bigger questions, the new questions we wanted to answer required contributions from across the Cores to address questions in a more comprehensive way,” said Weinfurt.

Many of the new questions that came up for the Core Working Groups originated from the NIH Collaboratory Trials.

“So many of the decisions about pragmatic trials involve compromise,” said Simon, chair of the Health Care Systems Interactions Core. “It is interesting how that compromise is reflected in terms of how and when the Cores work together because each Core has a focus but also needs to account for how an issue in one area of a trial also affects another area.”

For example, Simon said, a project might have a question related to working with the healthcare system, and that question may overlap with ethics and regulatory and healthcare access issues.

These overlapping issues have resulted in more collaborative work across Cores, explained Simon. “It is important to have these groups that have a focus as long as you realize that there are no sharp boundaries.”

The NIH Collaboratory’s Coordinating Center plays an important role in keeping track of the issues that arise from individual projects and following up with the Steering Committee and Cores.

“There is a web of information that is communicated through the project managers and communications specialists that allows a lot of issues to surface,” said Weinfurt. “Our NIH leaders will often raise issues that they are seeing either within the NIH Collaboratory or on the horizon as something that might be of interest to discuss.”

Looking to the future, Weinfurt said he expects to see more and more opportunities for the Cores to collaborate, find solutions, and create generalizable knowledge, especially as the new Community Health Improvement and Implementation Science Cores ramp up.

“Both the Community Health Improvement and the Implementation Science Cores have a similar reach across all of the various issues that come up for trials. It may be the case that it is very rare to have a Core work alone on a topic in the future,” said Weinfurt.

These collaborations have already been productive, as highlighted by recent cross-Core products tackling important issues:

October 24, 2022: NIH Pragmatic Trials Collaboratory Announces 2 New Core Working Groups

The NIH Pragmatic Trials Collaboratory is pleased to announce the addition of 2 new Core Working Groups that will be focused on implementation science and health equity.

Devon Check, PhD, and Hayden Bosworth, PhD
From left, Devon Check, PhD, and Hayden Bosworth, PhD

The Implementation Science Core will provide technical support and pragmatic trial expertise for NIH Collaboratory Trials with a specific focus on innovative dissemination and implementation science approaches. The Core will study methods and strategies to promote the uptake of interventions that have proven effective in routine practice, with the aim of improving population health.

The Implementation Science Core will be co-led by Devon Check and Hayden Bosworth. Devon Check, PhD, is a health services and implementation researcher and Assistant Professor, in the Department of Population Health Sciences at Duke University. Hayden Bosworth, PhD, is a health services and implementation researcher and Deputy Director for the Center for Health Services Research in Primary Care at the Durham VA Medical Center as well as Professor in the Department of Population Health Sciences and Professor of Medicine, Psychiatry, and Nursing at Duke University Medical Center. Jill George, CCRA, of the Duke Clinical Research Institute, will serve as the project manager for the Implementation Science Core in the Coordinating Center.

The Health Equity Core will provide leadership and guidance that will help the NIH Pragmatic Trials Collaboratory, its trials and investigators be more equitable in research. The Core’s work includes supporting pragmatic trials to address social and structural drivers of inequities, implement patient and community engagement strategies, promote the inclusion and mentorship of historically underrepresented scientists, and develop culturally and contextually aligned research and translation strategies that overcome bias and resonate with patients and communities.

Rosa Gonzalez-Guarda, PhD, MPH, RN, CPH, FAAN
Rosa Gonzalez-Guarda, PhD, MPH, RN, CPH, FAAN

The Health Equity Core will be led by Rosa Gonzalez-Guarda, PhD, MPH, RN, CPH, FAAN, an Associate Professor of Nursing at Duke University, Faculty Lead for the Population Health Research Area of Excellence at Duke’s Center for Nursing Research, and co-director of the Community Engagement Core for Duke Clinical Translational Science Institute. Alex Fist, MPH, of the Duke Clinical Research Institute, will serve as the project manager for the Health Equity Core in the Coordinating Center.

“By launching these 2 new Core Working Groups, the NIH Pragmatic Trials Collaboratory is reaffirming its commitment to generating knowledge that supports equitable pragmatic research to improve healthcare for all,” said Dr. Adrian Hernandez, co-principal investigator of the NIH Pragmatic Trials Coordinating Center. “Through their work in the areas of implementation science and health equity, the new Cores can help uncover how best to implement improved clinical practices so the benefits reach at-risk and traditionally underserved populations.”

Implementation Science

Implementation Science

Co-chairs:

Core goal: Support implementation-related research aims in pragmatic clinical trials to promote the uptake and sustainability of effective interventions in routine practice, with the aim of improving population health.

The Implementation Science Core will provide technical support and pragmatic trial expertise for NIH Collaboratory Trials with a specific focus on innovative dissemination and implementation science approaches. Improved dissemination and implementation of evidence-based practices requires the careful building of relationships with study teams early on. Integrating implementation frameworks and methods into the study design is essential for post-trial implementation success. The Implementation Science Core will meet regularly with NIH Collaboratory Trial study leaders from the start of their projects to help plan and work through any implementation related issues.

Pragmatic clinical trials have great potential to translate interventions into real-world practice. The Implementation Science Core will work to ensure that effective NIH Collaboratory Trial interventions achieve maximum impact, are scalable, and can be sustained in healthcare systems long-term.

Interview

During the NIH Pragmatic Trials Collaboratory 2024 Steering Committee meeting, Dr. Hayden Bosworth discussed the importance of planning for posttrial activities.

Areas of Focus

  • Support teams in thinking about appropriate implementation science theories, models, and frameworks to guide design and data collection for implementation aims.
  • Inform future implementation strategy design.
  • Identify measures for implementation constructs.
  • Enhance synergy between implementation research and pragmatic clinical trials.

News and Interviews


View More


Products and Publications


View More

Presentations


View More

Back to top

Community Health Improvement

Community Health Improvement

Co-chairs:

The Community Health Improvement Core provides leadership and guidance that will help the NIH Pragmatic Trials Collaboratory, its trials and investigators, and ultimately the research community at-large, conduct research that can improve the health of all populations. Their work includes supporting PCTs to address gaps in access to quality care, implementing patient and community engagement strategies, and developing person- and community-centered research and translation methods that are contextually aligned.

Core goals:

  • Support and document ways to help pragmatic trials be widely conducted across the United States in a variety of health systems and communities, including in rural areas.
  • Examine strategies to effectively and efficiently implement evidence-based interventions and guidelines across “real world” settings to improve the health of all populations.
  • Explore methods to make sure pragmatic research, including outcomes selected, is relevant to the people affected by the disease or condition being studied.
Image describing the elements that support health communities

Interview

During the NIH Pragmatic Trials Collaboratory 2024 Steering Committee meeting, Drs. Rosa Gonzalez-Guarda and Cherise Harrington discussed the importance of expanding the pragmatic clinical trials ecosystem to better serve minoritized communities.

Areas of Focus

  • Address gaps in healthcare access and quality through research design.
  • Offer guidance on how to ensure research questions are relevant to the people being served.
  • Generate new knowledge on PCTs that contribute to healthier communities.
  • Develop strategies to engage groups affected by trials conducted in healthcare systems.
  • Help researchers learn how to partner with communities and a wide variety of clinical sites.
  • Widely share new knowledge and resources for research that can have a positive impact on communities.

News and Interviews


View More


Products and Publications


View More

Presentations


View More

Back to top

Implementation Science

Implementation Science

Core goal: Support implementation-related research aims in pragmatic clinical trials to promote the uptake and sustainability of effective interventions in routine practice, with the aim of improving population health.

The Implementation Science Core will provide technical support and pragmatic trial expertise for NIH Collaboratory Trials with a specific focus on innovative dissemination and implementation science approaches. Improved dissemination and implementation of evidence-based practices requires the careful building of relationships with study teams early on. Integrating implementation frameworks and methods into the study design is essential for post-trial implementation success. The Implementation Science Core will meet regularly with NIH Collaboratory Trial study leaders from the start of their projects to help plan and work through any implementation related issues.

Pragmatic clinical trials have great potential to translate interventions into real-world practice. The Implementation Science Core will work to ensure that effective NIH Collaboratory Trial interventions achieve maximum impact, are scalable, and can be sustained in healthcare systems long-term.

Presentation

In this video, NIH Collaboratory
researchers Leah Tuzzio and
Dr. Vince Mor discuss what it
means to design a pragmatic
trial with implementation in
mind, and they share examples
from 2 NIH Collaboratory Trials—
ABATE Infection and PROVEN.

Areas of Focus

  • Support teams in thinking about appropriate implementation science theories, models, and frameworks to guide design and data collection for implementation aims.
  • Inform future implementation strategy design.
  • Identify measures for implementation constructs.
  • Enhance synergy between implementation research and pragmatic clinical trials.

News & Interviews


View More


Products and Publications


View More

Presentations


View More

Back to top

July 14, 2022: Solving Unexpected Challenges in Pragmatic Trial Data Collection

Headshot of Dr. Keith MarsoloDuring the NIH Pragmatic Trials Collaboratory Steering Committee meeting and 10th anniversary celebration, Dr. Keith Marsolo, Co-chair of the Electronic Health Records Core, moderated a panel on “Barriers and Challenges: Data Collection and Merging Datasets.” Four panelists shared their experiences collecting and aggregating data from diverse data sets and how they solved unexpected challenges.

The panelists included Drs. Ted Melnick (EMBED), Holt Oliver (ICD-Pieces), Margaret Kuklinski (GGC4H), and Andrea Cheville (NOHARM).

During the discussion, panelists shared lessons learned from collecting electronic health record (EHR) data during large pragmatic clinical trials across multiple health systems. Common challenges included:

  • Privacy concerns and not having the ability to collect deidentified patient or clinician demographic information
  • Working across health systems that use different EHR platforms and collecting data consistently across sites
  • Turnover of IT staff at participating sites and changes in site data agencies during the study period
  • Updates to the EHR that affect study algorithms and other issues with algorithms not identifying data as expected
  • Overcoming technical barriers with practice workflows and integrating with IT systems

The panelists shared solutions and possible best practices for future studies, including the need for planning, coordinating, and testing before study launch, the importance of being able to pivot and change directions as problems arise, being open to alternative data collection methods such as surveys to augment findings, and having the right team at the right time to be responsive to problems, which for one study meant having informaticists and expert EHR builders embedded in the trial team.

Panelists identified two key policy and infrastructure changes that would help trials be successful in the future. These changes include development of a national unique patient identifier across health systems and incentivizing EHR vendors to find common ground to better support research.

The Electronic Health Records Core continues to learn from experiences of the program’s NIH Collaboratory Trials and shares emerging information, resources, and EHR-related recommendations to improve future pragmatic research.

View slides from the discussion panel.

Learn more

View video collection that highlights advances in Electronic Health Records for pragmatic research.

Watch the August 2021 Interview with Electronic Health Record Core leaders Drs. Rachel Richesson and Keith Marsolo

June 7, 2022: Reflecting on 10 Years of the Health Care Systems Interactions Core

The NIH Pragmatic Trials Collaboratory Heath Care Systems Interactions Core supports and facilitates productive collaboration between researchers, clinicians, and health system leaders to conduct effective, relevant embedded pragmatic clinical research.

Health Care Systems Interactions Core Co-Chairs Dr. Eric B. Larson and Dr. Gregory Simon discussed the Core’s progress over the last 10 years in an interview at the NIH Pragmatic Trials Collaboratory Steering Committee meeting in April.

Over the last 10 years, the Health Care Systems Interactions (HCS) Core has allowed researchers to learn about working with healthcare delivery systems. Knowledge that is now common, was unknown when the Core was started, such as how dynamic healthcare delivery systems are and how the capabilities of and changes to the electronic health record can impact pragmatic clinical trials.

Through this type of discovery, the HCS Core has helped researchers become more sensitive to and aware of the priorities of healthcare delivery systems, resulting in better collaboration.

“The researchers’ priorities are usually not the same as the priorities of the people we are working with, whether they are patients, providers, or delivery systems. You have to know what other people’s purpose and drivers are and find a way to adapt,” said Larson. “We have learned and taken pretty seriously this idea of a learning health system with bidirectional engagement from research and from elements of the delivery system.”

The work of the HCS Core and NIH Pragmatic Trials Collaboratory has created a safe haven where researchers can share experiences and advance the field with common learning.

Simon sees the HCS Core as having internal and external missions. The internal mission is to support NIH Collaboratory Trials and be a community where researchers can come together, share their trials and tribulations, and experts in the Core can help these projects be successful, he described.

The external mission is focused on generalizable knowledge and advocacy. The HCS Core has shared knowledge with the research community and funders through publications and meetings and is advocating for research that includes the healthcare delivery system perspective.

“The discussion we are having is not just how do I work with healthcare systems to do my research, but how do I engage with healthcare systems about what research we should be doing, what are the right questions we should be asking for the studies that will be happening 5 years from now not the studies that are already underway,” Simon said.

Another lesson the HCS Core has learned is the importance of being flexible and adjusting. This lesson has been particularly relevant during the COVID-19 pandemic.

“COVID-19 is an extreme case of health system overwhelm, but I think we need to recognize that if we are going to serve people that have been traditionally not been well-served by the healthcare system, we will often be dealing with health systems that are chronically overwhelmed,” said Simon. “How do we do research in those settings? There are some really interesting challenges to think about.”

The HCS Core is focused on continued engagement between researchers and healthcare delivery systems that results in implementable new knowledge.

“My belief is that if we have the upstream involvement and are engaged in research projects that matter to the delivery system from the patients all the way up to the executives, we have a much better chance that when a result is valuable it becomes implementable and spreads to benefit everybody,” Larson said.

View the full interview.

See the complete materials from the 2022 Steering Committee meeting.

Patient-Centered Outcomes

Patient-Centered Outcomes

Co-chairs:

Patient-centered outcomes (PCOs) comprise measurable health outcomes that are important, impactful, and/or meaningful to patients. PCOs are comprised of variables collected directly from patients – patient reported outcomes (PROs) – and variables collected through other sources.

PCOs are comprised of variables collected directly from patients and variables collected through other sources

This figure shows a list of Patient Reported Outcomes. They are Health-related quality of life, symptoms, function, satisfaction with care or symptoms, adherence to prescribed medications or other therapy, and preceived value of treatment.Patient-reported outcome (PRO) data are defined by the FDA as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.” These data are increasingly used to inform and guide patient-centered care, clinical decision-making, and health policy decisions and are an important component of many of the Collaboratory’s NIH Collaboratory Trials.

Areas of Focus

The Patient-Centered Outcomes Core works closely with the Collaboratory to create guidelines and define best practices with respect to

  • Selecting, compiling, and curating the most appropriate PRO measures (and stimulating the development of new instruments when new solutions are needed).

  • Creating efficient, high-quality PRO data collection systems compatible with electronic health records and registries.

  • Conducting statistical analyses of PRO endpoints.

Interview

During the NIH Pragmatic Trials Collaboratory 2024 Steering Committee meeting, Drs. Emily O'Brien and Christy Zigler discussed challenges and lessons learned in collecting patient-centered outcome data in pragmatic clinical trials.

As part of this work, the Core conducts interviews with clinicians and other investigators using PROs, including NIH Collaboratory Trial principle investigators, to determine how PROs are currently used in the field, brainstorm solutions to challenges, and monitor progress with PRO implementation and data collection.

The Core has written a PROs Living Textbook chapter, a white paper Strategies for Overcoming Barriers to Patient-Reported Outcomes Measures, and a chapter on including PROs in registries as a part of the AHRQ handbook on registries. It also helped lead the development of an Effectiveness Guidance Document (EGD) for incorporating PROs into comparative effectiveness research in adult oncology and held a workshop to help clinicians implement the NIH Patient Reported Outcomes Measurement Information System (PROMIS).

News and Interviews


View More

Products and Publications

View More

Presentations

View More

Back to top

Ethics and Regulatory

Ethics and Regulatory

Co-chairs:

The advent of pragmatic clinical trials that leverage electronic health records and use research methods such as cluster randomization promises to answer critical research questions with increased sample sizes at lower costs, and could greatly increase production of the evidence needed to improve healthcare. This research, which relies on data collected as part of routine clinical care and may evaluate similar treatments in comparative effectiveness studies, poses a unique set of regulatory and ethical challenges. The Ethics and Regulatory Core identifies areas of regulatory and ethical uncertainty and works with the NIH Collaboratory Trials to navigate regulatory and ethical complexities associated with pragmatic clinical trials conducted within healthcare systems.

The Core is helping to develop approaches to the ethical design and conduct of pragmatic clinical trials that are in compliance with policies and regulations. Its activities supporting this work include the following:

  • Address common areas of concern for pragmatic clinical trials, such as minimal risk criteria and informed consent requirements

  • Liaise with regulatory and ethical oversight bodies, including institutional review boards (IRBs), the US Food and Drug Administration, and the Office for Human Research Protections

  • Conduct empirical research, when needed, to inform these issues

  • Document experiences and lessons learned to serve as a guide for other entities engaging in research within healthcare systems

An icon representing a computer monitor on which a virtual meeting is displayed.NIH Collaboratory Trial Ethics/Regulatory Discussions: Minutes and Supplementary Materials

The Core facilitates discussions on ethical and regulatory issues relevant to each of the NIH Collaboratory Trials. These discussions typically include representation from study principal investigators and study teams, members of the Ethics and Regulatory Core, NIH staff, and NIH Collaboratory Coordinating Center personnel, with additional input sometimes provided by representatives from the Office for Human Research Protections.

Areas of Focus

The Core convened groups of stakeholders to produce academic articles on a set of 11 ethical and regulatory issues related to the conduct of pragmatic clinical trials. The papers review the issues, and where possible, identify best practices and provide guidance for future pragmatic clinical trials. The following topics are addressed:

The set of eleven ethical and regulatory issues are informed consent, the distinction between research and quality improvement, FDA-regulated products, identifying direct and indirect subjects, data monitoring, vulnerable subjects, gatekeepers, defining minimal risk, IRB harmonization, the nature of intervention, and privacy.

Interview

During the NIH Pragmatic Trials Collaboratory 2024 Steering Committee meeting, Drs. Stephanie Morain and Pearl O'Rourke discussed ethics and regulatory challenges and lessons learned.

News and Interviews


View More

Products and Publications

View More

Presentations

View More

Back to top


Health Care Systems Interactions

Health Care Systems Interactions

Chair:

Core goal: Support and facilitate productive collaboration between researchers, clinicians, and health system leaders to conduct effective, relevant embedded pragmatic clinical research.

The Core aims to generate generalizable knowledge on how to engender trusting, long-term partnerships so stakeholders are eager to participate in research and trust that their interests, including providing excellent care to patients, will be well served. In addition, the Core shares knowledge on conducting research in a way that does not disrupt the delivery of clinical care or health systems’ operations.

The Core works with the Collaboratory to identify and consult on challenges, provide a forum for discussion, and develop tools and best practices for research conducted in partnership with diverse healthcare systems. The Core’s activities are applicable throughout the life cycle of clinical trials.

Interview

During the NIH Pragmatic Trials Collaboratory 2024 Steering Committee meeting, Dr. Greg Simon discussed challenges and lessons learned in healthcare system engagement.

Areas of Focus

This figure shows that a project moves from the protocol development stage to study start-up then on to study conduct and finally ends with study closeout. Along the way, these steps require collaboration, developing best practices, engagement, communication, and knowledge-sharing.

This page also includes products from the Stakeholder Engagement Core led by Sean Tunis, CMTP, from 2012-2017.

News and Interviews


View More

Products and Publications


View More

Presentations


View More

Back to top