The table below shows the range of stakeholders that might participate in a PCT. Not every stakeholder is relevant to every PCT or at every stage of the research, but identifying these at the outset will set the stage for effective engagement. One approach to determining which categories of stakeholders are important for a particular PCT is to consider the following questions: 1) Who can help minimize potential barriers to study completion? and 2) Who will use the evidence from the study to make decisions or be affected by those decisions? For PCTs conducted in the context of healthcare delivery, healthcare delivery organization leaders, clinicians, and patients will always be important stakeholder groups (Moloney et al. 2016).
Potential PCT Stakeholders
|Patients, caregivers, and consumer advocacy groups||Current and potential consumers of healthcare, their caregivers, families, and patient and consumer advocacy groups|
|Clinicians||Physicians, nurses, mental health professionals, pharmacists, paramedics, and other providers of care and support services|
|Healthcare delivery organization leaders||Chief executive officers, chief financial officers, chief operations officers, chief medical officers, directors, and other executive-level leaders or senior management within health systems, hospitals, skilled nursing facilities, and other healthcare delivery organizations|
|Operational personnel||Operational managers, IT, billing, compliance, and other business operations staff|
|Payers and purchasers||Private insurers, Medicare, Medicaid, employers, the self-insured, and state, government, and other entities responsible for reimbursing or underwriting the costs of healthcare|
|Policy-makers and regulators||Department of Health and Human Services (e.g., US Food and Drug Administration, Office for Human Research Protections), Congress, the White House, states, professional associations, and other regulating or policy-making entities and their intermediaries|
|Research funders||Government and private funders of research|
|Researcher||Academic, industry, clinical, or patient investigator with a question|
|Product manufacturers||Manufacturers of drugs and medical devices, electronic health record vendors|
|Medical Societies||Medical Societies, such as the American College of Surgeons, may generate guidelines and best practices and may help disseminate findings.|
Adapted from (Concannon et al. 2012)
For researchers who are not part of a large academic organization or who have not partnered with stakeholders before, deciding who, when, and how to engage and can be a challenge. The solution will be different depending on the particular scenario, but a good place to start is with someone who can champion the project and provide a warm introduction to the stakeholder. Other ways to find stakeholders is by attending conferences and networking, and by identifying learning health systems that value doing partnered pragmatic research.
After individuals have been identified, there are many different ways to engage them:
- Advisory groups: The STOP CRC Demonstration Project collaborated with an existing patient advisory council in some of their participating health centers to get feedback on the materials they used (wordless instructions for colorectal cancer screening kits). The PIs also formed an advisory board with a representative from each center, payers, and researchers.
- “You don’t know what kind of magic will happen when they are all in the same room together.”—Gloria Coronado, PhD
- Advisory boards can include healthcare delivery organization leaders, clinicians, operations personnel, patients, caregivers, patient advocacy groups, payers and purchasers, policymakers and regulators, research funders, researchers, and product manufacturers.
- Collaborative pitching and co-design: Involving stakeholders from the very beginning can produce strong partnerships and champions, and will help identify values, priorities, and perspectives that will be important throughout the research continuum.
- IDEO.org is a group that focuses on “human-centered design” and offers free resources about different ways to engage stakeholders.
Healthcare Delivery Organization Leaders
Because PCTs are typically conducted using information in the electronic health record (EHR) and as part of routine care, they could not occur without the partnership and buy-in of a healthcare delivery organization. To illustrate the importance of health system leader engagement, we gathered healthcare systems leaders for a panel modeled after the Shark Tank TV show. In the video, Dr. Greg Simon is tasked with convincing healthcare system leadership to invest in implementing the intervention from the Suicide Prevention Outreach Trial (SPOT).
Panelists (or sharks) include:
- Susan Mullaney, President of Kaiser Foundation Health Plan of Washington
- Edward Septimus, formerly Vice President for Research and Infectious Disease at HCA Healthcare and currently a Clinical Professor at Texas A&M Medical School
- Matt Hough, Medical Director of Jackson Care Connected Care, Oregon
Several national initiatives have been involved in investigating challenges and practical strategies for integrating research in the setting of clinical care (Institute of Medicine 2013). According to a 2014 survey, health system executives are interested in research studies that support organizational performance goals; provide data to drive decision-making; enhance delivery-system reputation and national and community connections; and ultimately support the goal of high-quality, patient-centered care at a reasonable cost (Institute of Medicine 2015; Larson and Johnson 2015). Leaders were enthusiastic regarding the prospect of integrating knowledge generation into care, but wanted to minimize the impact of the research process on clinical operations and improve the speed and availability of research results. Healthcare delivery organization leaders are often gatekeepers, defined as “people or entities who can allow or deny access to resources required to support the conduct of clinical research.” (Patterson et al. 2011; Whicher et al. 2015). These individuals play a critical role in setting up an effective context to test the trial and in determining which PCTs are implemented and if the results of the trial become routine care. Aside from the essential early engagement with healthcare delivery organization leaders, operational personnel can offer insight crucial to the success of a PCT, such as an understanding of existing infrastructure and clinical workflows.
A lesson emphasized by Gregory Simon, MD, principal investigator of one of the NIH Collaboratory Demonstration Projects is, “researchers often have a tail-wagging-the-dog problem… we need to remember that we’re the tail and the healthcare system is the dog” (see full interview).
Even a highly developed and centralized healthcare delivery infrastructure does not obviate the need for local-level engagement with front-line clinicians and staff to facilitate successful trial completion (Tunis et al. 2016). Collaboratory investigators have cited the value of clinician engagement and how it is an important challenge to conducting PCTs. Overall, PCT researchers should have a communication plan, design training approaches that can be available on demand (e.g., recorded webinars) and updated as needed, be prepared to learn from their health system partners, and be flexible, adapting as needed to the dynamic study environment. Re-training and re-engagement will be needed due to staff turnover.
Some of the key issues with regard to engaging clinicians are described in the Grand Rounds, “Straight from the Source: Clinicians’ Views on Participating in CER/PCOR,” given by Ellen Tambor MA, Rachel Moloney, MHS, and Sean Tunis, MD.
- Qualitative, empirical evidence pertaining to clinician participation in comparative effectiveness and patient-centered outcomes research (CER/PCOR) is very limited.
- Literature has shown that the biggest motivations for clinician involvement in research are improving patient care and contributing to clinical knowledge, rather than recognition or financial motives.
- Early and ongoing engagement builds clinician trust, enthusiasm, confidence, and commitment in pragmatic trials.
- Focus groups with clinicians made it clear that clinicians want to be involved in early stages of study design and want to protect their patients from feeling like “guinea pigs.”
Including patients as partners throughout the research process through meaningful engagement can help PCTs be more patient centered, and may be a requirement for some research funding. Patient representatives might include individuals with lived experience of the disease or condition in question, family members or caregivers, or representatives from patient or consumer advocacy organizations. Decisions about who to include should be based on the topic and stage of research, the role patient representatives will be asked to play, the type of input that is needed, and the types of skills or experience required to participate meaningfully, among other factors. If relationships with patients and advocates do not already exist, advocacy organizations relevant to the disease or condition under study may be able to help identify patients who would be willing to serve as representative, or alternatively, to appoint a representative from their organization. Clinicians involved with the trial can also help to identify patient representatives. The Colorado Patient Partners in Research Campaign, a partnership between Kaiser Permanente and a local hospital, created a web-based system for matching patient partners with research teams in Colorado; they sent emails and launched a social media campaign to drive individuals to the website. PCT researchers should be prepared to build a trusting relationship, and to learn from, train, and provide time and space to collaborate with patient representatives.
A panel funded by the Patient-Centered Outcomes Research Institute (PCORI) published recommendations for the oversight of patients who participate in research roles other than as “research participant" (Gelinas et al. 2018). When patients and caregivers participate in roles such as co-investigators, study personnel, and advisors in research studies, novel ethical and regulatory challenges could develop. The panel provides a taxonomy for these roles and recommendations for appropriate oversight. The group also provides recommendations about identifying and engaging a diverse mix of patients and developing mechanisms to protect against possible conflicts of interest.
In an accompanying editorial, Dr. Robert Califf expressed his support for the panel’s efforts and their taxonomy for patients in patient-centered outcomes research:
“The differences among being a subject of research, offering advice or perspectives about a research project, and playing a role as a researcher are important and deserve special consideration in the context of ethical and oversight frameworks governing research.” — (Califf RM. 2018)
Other potential stakeholders include healthcare payers, policy-makers, and guideline developers who rely on the evidence from clinical trials to inform decisions that may affect large populations of patients and consumers. This case study from the Strategies and Opportunities to Stop Colorectal Cancer (STOP CRC) in Priority Populations trial, illustrates one example of engaging with payers.
It may also be appropriate to confer with policy-makers, guideline developers, and medical societies to see if any new guidelines are slated for release during the conduct of the trial that may impact the results or outcomes of the PCT. Additionally, one can partner with these stakeholders to determine what evidence is needed to support future guidelines. For example, Trauma Survivors Outcomes and Support Trial (TSOS) was developed in partnership with American College of Surgeons Committee on Trauma so the findings from pragmatic trials could be integrated into guidelines that regulate trauma care nationally. (For more, see the case study).
Enlisting the involvement of these stakeholder groups can be made easier if the PCT addresses questions that are of particular relevance. In some cases, it may also be useful to engage with relevant product manufacturers, consumer advocacy groups, or professional societies. However, in all cases, care should be taken to ensure that the interests of one stakeholder do not have undue influence on the research process.
Grand Rounds March 16, 2018: Straight from the Source: Clinicians’ Views on Participating in CER/PCOR (Ellen Tambor MA, Rachel Moloney, MHS, and Sean Tunis, MD)
Podcast March 22, 2018: Straight from the Source: Clinicians’ Views on Participating in CER/PCOR (Ellen Tambor MA, Rachel Moloney, MHS, and Sean Tunis, MD)
Califf RM. 2018. A Beginning to Principles of Ethical and Regulatory Oversight of Patient-Centered Research. Ann Intern Med. 169(8):579-580. doi: 10.7326/M18-2517. PMID:30264090.
Concannon TW, Meissner P, Grunbaum JA, et al. 2012. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 27:985–991. doi:10.1007/s11606-012-2037-1. PMID: 22528615.
Gelinas L, Weissman JS, Lynch HF, et al. 2018. Oversight of Patient-Centered Outcomes Research: Recommendations From a Delphi Panel. Ann Intern Med. 169(8):559-563. doi: 10.7326/M18-1334. PMID:30264127.
Institute of Medicine. 2013. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Smith M, Saunders R, Stuckhardt L, McGinnis JM, editors. Washington, DC: National Academies Press. https://www.nap.edu/catalog/13444/best-care-at-lower-cost-the-path-to-continuously-learning. Accessed May 9, 2017.
Institute of Medicine. 2015. Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. Washington, DC: National Academies Press. https://www.nap.edu/catalog/18945/integrating-research-and-practice-health-system-leaders-working-toward-high. Accessed May 9, 2017.
Larson E, Johnson K. 2015. Making new care models a reality requires closer collaboration between researchers and execs. Modern Healthcare. http://www.modernhealthcare.com/article/20150822/MAGAZINE/308229977. Accessed 2017 May 9.
Moloney RM, Tambor ES, Tunis SR. 2016. Patient and clinician support for the learning healthcare system: recommendations for enhancing value. J Comp Eff Res. 5:123–128. doi:10.2217/cer.15.67. PMID: 26930026.
National Institute for Health Research. 2016. The James Lind Alliance Guidebook. www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-Version-6-February-2016.pdf. Accessed May 9, 2017.
Patterson S, Mairs H, Borschmann R. 2011. Successful recruitment to trials: a phased approach to opening gates and building bridges. BMC Med Res Methodol. 11:73. doi:10.1186/1471-2288-11-73. PMID: 21595906.
Tunis S, Tambor E, Moloney R. 2016. Clinician Engagement in the NIH Collaboratory and Beyond. PCT Grand Rounds presentation; May 6, 2016.
Whicher DM, Miller JE, Dunham KM, Joffe S. 2015. Gatekeepers for pragmatic clinical trials. Clin Trials. 12:442–448. doi:10.1177/1740774515597699. PMID: 26374683.