July 16, 2019: ACP PEACE Trial Moves From Planning to Implementation Phase: An Interview With Dr. Angelo Volandes

Dr. Angelo Volandes
Dr. Angelo Volandes

The National Institute on Aging (NIA) recently approved the Advance Care Planning: Promoting Effective and Aligned Communication in the Elderly (ACP PEACE) trial, an NIH Collaboratory Demonstration Project, to move from the planning phase to the implementation phase. The goal of ACP PEACE is to evaluate a comprehensive advance care planning program that combines clinician communication skills training and patient video decision aids.

We spoke with Dr. Angelo Volandes, co–principal investigator of ACP PEACE with Dr. James Tulsky, at the NIH Collaboratory Steering Committee meeting in May about what the study team has learned during the planning phase of the trial.

Were there surprises during the planning phase of the study?

There were lots of surprises. The biggest surprise was that most clinicians don’t use the structured variable in the electronic health record (EHR) that we were going to use to extract our primary outcome. The workaround, which I think is actually better, is to use natural language processing (NLP) to abstract our primary outcome from the free text of the clinical note in the EHR.

The other big surprise was that oncologists really enjoyed the intervention. They have been open to the skills training and, if anything, they’ve asked for more.

What is an example of a challenge you were able to overcome with the help of the NIH Collaboratory Core Working Groups?

One challenge we encountered is related to an issue discussed in a paper by NIH Collaboratory investigators Dr. Kevin Weinfurt and Dr. Jeremy Sugarman and colleagues. It has to do with the idea of “broadcast notification.” One of our 3 participating healthcare systems asks patients if they will allow their deidentified medical record data to be used for research purposes. Every patient in that healthcare system has the option to opt out of having their deidentified data used for research purposes. Our other 2 participating healthcare systems don’t do that as a routine matter. So we needed a different approach.

Dr. James Tulsky
Dr. James Tulsky

The idea of broadcast notification—which is new and was developed in the NIH Collaboratory—is to display posters or other notices in healthcare settings that let patients know they can opt out if they have a concern about their deidentified data being shared for research purposes. Our local institutional review board (IRB) was unfamiliar with this approach. Having the Ethics and Regulatory Core help us understand the approach and educate our IRB was incredibly helpful. It was especially helpful to be able to share a published, peer-reviewed paper showing that this was an issue the NIH Collaboratory had studied.

(Editor’s note: Read the article by Weinfurt et al,  “Comparison of Approaches for Notification and Authorization in Pragmatic Clinical Research Evaluating Commonly Used Medical Practices,” in the November 2017 issue of Medical Care.)

What other key challenges have you faced?

There are always competing priorities in real-world oncology clinics. For example, there are quality improvement projects all over the place. When you’re the clinician, how do you devote the appropriate attention and time to this particular project? We feel our project is at the crux of patient-centered care, about understanding the goals, values, and beliefs of patients when it comes to serious illness care. But there are competing priorities. There can be a tension between the time you need to get the project done, for the intervention to truly reach its fruition, versus what a clinic might be willing to do.

What advice do you have for investigators conducting their first embedded pragmatic clinical trial (ePCT)?

It’s really important to get the appropriate buy-in from people in high enough positions of authority so that the project happens. It is not enough to get the chief research officer of a healthcare system to say the project is a great idea. You need the chief marketing officer, the chief executive officer, the finance people to sign off on it. When you’re in the pragmatic research world, it’s no longer just research in a controlled environment. It affects things you didn’t think about—like patient flow, revenue—and everything has to be accounted for. Make sure you get appropriate buy-in from enough stakeholders to know that you’re going to get the project done.

Also, don’t underestimate the costs of information technology (IT). For example, we need a lot more resources for our IT infrastructure now that we have switched from using a structured variable to using NLP to obtain our primary outcome. Make sure you have thought through IT needs, especially in pragmatic trials, where so much is abstracted from the EHR. Think carefully, early on, about how much time you will need from the IT group.

Anything else you want to say about ePCTs or the NIH Collaboratory?

It’s critically important to participate in the regular meetings of the Core Working Groups, to take advantage of them to help you address challenges. For example, when we encountered the problem with obtaining the primary outcome, we presented it to the EHR Core. When we had the challenge with notification and the IRB, we presented it during a meeting of the Ethics and Regulatory Core. The Core Working Groups are most useful when you openly share the challenges you are facing. That’s the way to get help from the Cores. This is my second pragmatic trial, and I’m not perfect. I put it out there because I want help from the experts.

ACP PEACE is supported within the NIH Collaboratory by a cooperative agreement from the NIA and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about ACP PEACE in the Living Textbook, and learn more about the NIH Collaboratory Demonstration Projects.

July 15, 2019: PRIM-ER Gains Approval to Proceed to Implementation Phase: An Interview With Dr. Corita Grudzen

Primary Palliative Care for Emergency Medicine (PRIM-ER)Dr Corita Grudzen, an NIH Collaboratory Demonstration Project, received approval this month to enter its implementation phase. PRIM-ER is a pragmatic, cluster randomized trial of a multidisciplinary primary palliative care education and decision support intervention in a diverse sample of emergency departments that differ in their specialty geriatric and palliative care capacity, geographic region, payer mix, and patient demographic characteristics.

We spoke with the principal investigator of PRIM-ER, Dr. Corita Grudzen, at the NIH Collaboratory Steering Committee meeting in May about the completion of the study’s planning phase.

What is the status of your study?

PRIM-ER is currently transitioning from a 1-year UG3 planning phase to the UH3 implementation phase, where the intervention will be tested at full scale across 33 sites in a stepped-wedge trial. Our program manager is conducting training sessions in preparation for implementing the intervention at study sites.

Were there surprises during the planning phase of the study?

There weren’t surprises so much as confirmation of what we already knew, which was that enthusiasm can go a long way. Having a detail-oriented, responsive site principal investigator is also key. When you have a well-organized team on the ground, the distance doesn’t matter as much, and all the things you think are going to get in the way don’t get in the way. Things go smoothly when you have someone who is incredibly enthusiastic, loves the content, understands the content, and is infectious about sharing the training and information with others. I think that can overcome a lot of potential barriers.

What is an example of a challenge that you were able to overcome with the help of the NIH Collaboratory Core Working Groups?

Working with the Cores was reassuring, in that they showed us we weren’t alone. We were all struggling with the same issues, and it was okay not to be perfect in the way we were attacking all the problems. We were all planning pragmatic trials, and it was okay if we showed our warts and all.

What other challenges have you faced?

Having a good team at the primary organization is really important. It’s important to have a good administrative team—a program manager, project director—who can hold down the fort, especially with pragmatic trials, when you’re traveling to a lot of sites.

What advice do you have for investigators conducting their first embedded pragmatic clinical trial?

Give yourself a break. There are a ton of imperfections in conducting embedded pragmatic trials. It’s all about the people. Pick great site principal investigators. That’s more important than anything else about the institutions. You want to have enough eligible patients that you’re going to have an impact or whatever else is involved in picking your sites. Enthusiasm and organizational savvy go a long way. Be patient and flexible and open to new iterations of what you’re doing. It feels scary at first, but I think it will serve you to be open to change.

PRIM-ER is supported within the NIH Collaboratory by cooperative agreements from NCCIH and the National Institute on Aging and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about PRIM-ER and the NIH Collaboratory Demonstration Projects.

July 3, 2019: New Article Describes the Public’s Beliefs Regarding Responsibility to Participate in Research

Findings from a new article suggest that the majority of patients do not feel a personal responsibility to participate in clinical research. In the article, Kevin Weinfurt, Li Lin, and Jeremy Sugarman report the results of a national survey of nearly 3000 people regarding their attitudes towards research responsibilities as well as their trust in doctors, healthcare systems, and medical research. Ethical frameworks for learning health systems have suggested that patients have a responsibility to contribute to learning activities, including research. The findings from this survey suggest that most patients in the U.S. do not currently endorse such a responsibility.

“These data provide a useful snapshot of the public’s views toward the obligation to participate in research. It is unclear how, if at all, these views will shift with increased efforts to create mature learning health systems. And if such views do not shift, it is uncertain what that would mean for the success of learning health systems.” —Kevin Weinfurt, PhD

Read the full article: Public Views Regarding the Responsibility of Patients, Clinicians, and Institutions to Participate in Research in the U.S.

For more on alternate approaches to consent, see the Living Textbook Chapter on Consent, Disclosure, and Non-Disclosure

July 2, 2019: New Living Textbook Section on Inpatient Endpoints in Pragmatic Clinical Trials

A new section in the Living Textbook describes the considerations for using “real-world” data for inpatient-based event ascertainment. There are many sources for acquiring this information, and they have different time lags in their availability and varying degrees of error and bias. In order to use inpatient endpoints in pragmatic clinical trials, these factors must be understood during the design, conduct, and analysis phases of an embedded pragmatic clinical trial.

“The pragmatic trial community needs to collectively determine which endpoints are relevant for pragmatic trials, how they can be measured and validated, and how the accuracy of these measurement methods may impact hypothesis testing sample size estimates.” —Eisenstein et al 2019

Topics in the chapter include:

  • Pragmatic trial inpatient endpoints
  • Inpatient event data sources
  • Patient-reported data
  • Secondary data sources: EHR
  • Secondary data sources: claims
  • Case studies: ICD-Pieces, TRANSFORM-HF, ADAPTABLE, and TRANSLATE ACS
  • Data source accuracy

 

Podcast June 26, 2019: Good Clinical Practice Guidance and Pragmatic Trials: Balancing the Best of Both Worlds in the Learning Health System (Robert Mentz, MD)

June 25, 2019: GGC4H Awarded Continuation From Planning to Implementation Phase

The investigators of Guiding Good Choices for Health (GGC4H), an NIH Collaboratory Demonstration Project, have received approval to move from the planning phase to the implementation phase of their study. Congratulations to the GGC4H study team for their excellent work!

At the May 2019 NIH Collaboratory Steering Committee meeting, we talked with Drs. Stacy Sterling, Margaret Kuklinski, and Richard Catalano to hear about progress and challenges during the UG3 planning phase. The goal of GGC4H is to test the feasibility and effectiveness of implementing Guiding Good Choices—a universal evidence-based anticipatory guidance curriculum for parents of early adolescents—in 3 large, integrated healthcare systems serving socioeconomically diverse families.

“Guiding Good Choices is a tested and proven intervention in community and school-based settings. We think primary care is an ideal implementation setting because pediatricians have parents’ trust, and the AAP recommends that they offer anticipatory guidance to parents. Community and school settings do not often have the same advantages.” — Rico Catalano, Co-PI of GGC4H

In the UG3 phase, the study team partnered with 5 pediatric primary care clinics in Kaiser Permanente Northern California, Kaiser Permanente Colorado, and Henry Ford Health System, finalized the trial protocol, obtained IRB approval, and presented UG3 and pilot study findings at 2 recent national meetings: the Society for Prevention Research and the National Academy of Medicine Collaborative for Healthy Parenting in Primary Care.

Were there any surprises during the study’s planning phase?

“I’ve been pleased and surprised to see the near universal excitement for Guiding Good Choices across the 3 healthcare systems and in multiple clinics. My previous work has been with adolescents who have already started to develop problems and are clearly at risk. Often when you are engaging those teens and their parents, they are coming from a place of crisis and have a lot of anxiety and worry. With Guiding Good Choices, we’re offering it to all families of young adolescents in the pediatric clinics, and we’re heartened to see pediatricians and parents welcoming this.

“The parents and kids were so enthusiastic and excited in our pilot. It’s been extremely heartening, and I think it bodes well for the reception we’re going to get.” — Stacy Sterling, Co-PI of GGC4H

What is an example of a challenge that you were able to overcome with the help of a Core Working Group?

Our study has two core questions: Will Guiding Good Choices improve adolescent behavioral health when offered in a health care setting? Will parents in a health care setting actually enroll in Guiding Good Choices and to what degree? Our initial study design attended more to the second question, and in doing that, raised problems for the valid assessment of effectiveness. These issues could have prevented us from transitioning to the implementation phase, so we needed a good design for assessing both effectiveness and implementation. Our original plan would have included all adolescents who had well visits, but at some clinics, 25% of teens don’t have them. Our new design, developed with the IRB chair and the Biostatistics and Study Design Core, includes everyone who receives care at the pediatric clinic. Although we may enroll some people who don’t engage with the intervention, this will make the study results more generalizable and valid for both effectiveness and implementation.

What words of advice do you have for investigators conducting their first embedded PCT?

“It’s a really fasted-paced year. I think we have complementary strengths across the leaders at our sites, and that is important. Pay attention to the qualities of your team and how they can help you hit the ground running and eventually help you get across the finish line to the UH3 phase.”  — Margaret Kuklinski, Co-PI of GGC4H

GGC4H is supported within the NIH Collaboratory by a cooperative agreement from the National Center for Complementary and Integrative Health and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about GGC4H in the Living Textbook, and learn more about the NIH Collaboratory Demonstration Projects.

June 20, 2019: EMBED Investigators Discuss Progress and Transition to Implementation Phase

At the May 2019 meeting of the NIH Collaboratory Steering Committee, we talked with Drs. Ted Melnick and Gail D’Onofrio of EMBED, an NIH Collaboratory Demonstration Project, to hear about progress and challenges during the UG3 planning phase. The goal of EMBED is to test whether implementation of a user-centered clinical decision support system increases adoption of initiation of buprenorphine/naloxone into the routine emergency care of patients with opioid use disorder. In the UG3 phase, the study team put in place the infrastructure of a pragmatic, multicenter, parallel, group-randomized health IT intervention. EMBED recently transitioned to the UH3 implementation phase and plans to launch the intervention at 20 sites across 5 healthcare systems in August 2019.

“With EMBED, we’re trying to take evidence-based research and implement it to improve practice. EMBED is both a research and patient care project.”

Were there any surprises during the study’s planning phase?

The first surprise came at last year’s Steering Committee meeting, when we met with the Biostatistics and Study Design Core. They encouraged us to change our original study design from stepped-wedge to group-randomized, which we did. We think this advice led to a stronger study. The main reason for this is the group-randomized design’s ability to better account for temporal changes. Since our intervention is being conducted in the middle of an opioid crisis, there are potentially other concurrent interventions that could make it difficult to determine the effect of our intervention. The group-randomized design should give us better insight into whether our intervention is driving behavior change in treating patients with opioid use disorder.

What is an example of a challenge that you were able to overcome with the help of a Core Working Group?

In addition to design advice from the Biostatistics Core, we received expert guidance from the Ethics and Regulatory Core, who helped us prepare for the central IRB process. The Core’s input was essential to how we developed our protocol’s waiver of informed consent, data handling, and protection of patient privacy. We were able to demonstrate to the IRB that our approach was logical and informed. We think this helped the IRB “get it” and allowed us to more efficiently address patient privacy issues in a vulnerable population across multiple healthcare systems.

What other key challenges have you faced?

One challenge was on the IT side with electronic health record (EHR) integration, which required more customization than we initially planned. How we work with EHR vendors is evolving, and we’ve found good partners so that we can integrate across different systems. This has strengthened our intervention so that it is perceived as more universal than one designed only for a specific EHR system.

Another challenge is the general under-resourcing of healthcare delivery systems for pragmatic research. We found that, regardless of budget, getting approval from system leadership for an IT change is often not enough—what is needed is figuring out who is going to make the change, how much time is involved, and whether the team has the bandwidth to complete the task. You cannot underestimate the degree of difficulty a change poses to a health system that is still struggling to get the clinical side of things right.

The way a study is framed to leadership is important—understand what’s motivating them to participate and move a project forward. With EMBED, we’re trying to take evidence-based research and implement it to improve practice. EMBED is both a research and patient care project. We need to impress upon leadership that we can improve patient outcomes and we’ll pay for it, but we need their help and support in navigating the process through the institution.

What words of advice do you have for investigators conducting their first embedded PCT?

  • Study teams should think about potential barriers from the beginning and find solutions quickly.
  • Make sure that health system leadership discusses your project with those on the ground.
  • Enlist the experts your study needs for each site. In our case, we needed both an IT expert for the operational side and a clinical expert, or we couldn’t have moved the project forward.
  • Recognize that there are trade-offs in pragmatic design and remember that you’re working with health systems in which your intervention will need to be replicated.
  • Make your intervention sustainable and easily usable by the clinician, without the need for research or other additional staff.

EMBED is supported within the NIH Collaboratory by a cooperative agreement from the National Institute on Drug Abuse and receives logistical and technical support from the NIH Collaboratory Coordinating Center. Read more about EMBED in the Living Textbook, and learn more about the NIH Collaboratory Demonstration Projects.

June 13, 2019: Experience With Pragmatic Clinical Trials Gains Momentum

At the NIH Collaboratory Steering Committee Meeting in May 2019, participants shared their perspectives on the evolving landscape of embedded pragmatic clinical trials (ePCTs). Three initiatives were presented: the Patient-Centered Outcomes Research Institute (PCORI), the NIH-DoD-VA Pain Management Collaboratory, and the HEAL (Helping to End Addiction Long-term) Initiative. Although many challenges remain, the conduct of ePCTs is gaining momentum, and the synergy between the initiatives, along with the fellowship they engender, will continue to help pave the way for more embedded pragmatic research in the future.

Dr. Ann Trontell, Associate Director of Clinical Effectiveness and Decision Science at PCORI, shared PCORI’s experience with pragmatic clinical studies. Since 2014, PCORI has awarded $494 million dollars for 43 pragmatic studies that range in size from 425 to 100,000 participants (median, approximately 1700). The studies include 2 observational, 27 individually randomized, and 14 cluster randomized trials in a wide range of therapeutic areas.

Dr. Trontell urged those developing proposals for pragmatic trials to make them fit for purpose, as opposed to emphasizing pragmatism, a theme echoed in the Developing a Compelling Grant Application chapter of the Living Textbook.

 

 

 

 

Dr. Robert Kerns, a director of the NIH-DoD-VA Pain Management Collaboratory, shared progress with pragmatic trials designed to evaluate whether evidence-based nonpharmacological approaches are effective for pain management among US military personnel and veterans.

Modeled after the NIH Collaboratory, the Pain Management Collaboratory is supporting 11 projects through a 2-year planning phase and a 2- to 4-year implementation phase. Subject matter experts at the Pain Management Collaboratory Coordinating Center (PMC3) support the projects by sharing tools, best practices, and resources.

 

Dr. Wendy Weber, Program Officer for the NIH Collaboratory Coordinating Center, introduced the HEAL initiative, which is designed to enhance pain management and improve prevention and treatment strategies for opioid misuse and addiction. The goal of the initiative is to provide scientific solutions to the opioid crisis. It includes a set of large-scale pragmatic trials that will receive logistical and technical support from the NIH Collaboratory Coordinating Center.

 

While experience with ePCTs is growing, many distinct challenges remain. As the conduct of ePCTs gains momentum, there is a rich opportunity to use collective experiences to refine best practices to real-world evidence generation and help solve urgent public health problems.

June 11, 2019: NIH Collaboratory Hosts Daylong ePCT Seminar at AcademyHealth Annual Research Meeting

AcademyHealthNearly 50 researchers attended a preconference seminar at the 2019 AcademyHealth Annual Research Meeting in Washington, DC, to learn the essentials of embedded pragmatic clinical trials (ePCTs). The full-day seminar, offered by the NIH Collaboratory in partnership with AcademyHealth, introduced attendees to concepts in the design, conduct, and implementation of ePCTs, with a particular focus on methods relevant to health services researchers.

Investigators from the NIH Collaboratory Coordinating Center, Core Working Groups, and Demonstration Projects, along with officials from the NIH and the Office of Human Research Protections, led attendees through a dozen modules. Topics included considerations in the design and implementation of ePCTs, engaging with health system partners and other stakeholders, case studies from the NIH Collaboratory Demonstration Projects, and more.

“As the Collaboratory continues to generate evidence and develop new approaches, seminars like this one allow us to pass that knowledge on to new audiences of investigators and extend its reach and impact,” said Dr. Kevin Weinfurt, a co–principal investigator of the NIH Collaboratory Coordinating Center who led the seminar.

The preconference seminar was a condensed version of a 2-day training workshop offered by the NIH Collaboratory in February 2018. Access the full set of training resources in the Living Textbook.

June 7, 2019: Meeting Materials from the 2019 NIH Collaboratory Steering Committee Meeting

The Collaboratory has made available all the presentations from their recent Steering Committee meeting held in Bethesda May 1-2, 2019.

Highlights of Day 1 included updates on the progress and sustainability of the NIH Collaboratory, perspectives on the landscape of embedded PCTs (ePCTs) and the need for real-world evidence, challenges and lessons learned from the UH3 Demonstration Projects, updates on progress and transition plans from the UG3 Demonstration Projects, and discussions on data sharing policy and planning. Day 2 featured an intensive workshop hosted by the NIH with the goal of starting discussions on statistical issues with ePCTs.

View or download the meeting materials on the website.