Dissemination & Implementation in Embedded Pragmatic Trials: Raising the Bar for Real-World Research: AcademyHealth (December 2023)

December 10, 2023: The NIH Pragmatic Trials Collaboratory hosted a Workshop at the 2023 AcademyHealth 16th Annual Conference on the Science of Dissemination and Implementation in Health. This training workshop introduces concepts in the design, conduct, and implementation of embedded pragmatic clinical trials (ePCTs), and provides firsthand ePCT experiences and case studies from the NIH Pragmatic Trials Collaboratory Trials.

Agenda and Slides

Morning Sessions

Welcome and Introduction

Emily O'Brien, PhD

Session 1: What Are Embedded Pragmatic Clinical Trials?
Emily O'Brien, PhD

Session 2: Objectives and Trial Design: An Overview of Hybrid Designs
Hayden Bosworth, PhD

Session 3: Engaging with Health System and Community Partners
Devon Check, PhD

Session 4: ePCTs in Context: Small Group Work Followed by Panel Discussion with NIH Collaboratory Trials PIs

Moderator: Emily O'Brien, PhD

*This session includes case study presentations.

Afternoon Sessions

Session 5: Measuring Outcomes
Christy Zigler, PhD, MSEd

Session 6: ePCT Design
Jonathan Moyer, PhD

Session 7: ePCT Analysis
Jonathan Moyer, PhD

Session 8: Pilot & Feasibility Testing
Beda Jean-Francois, PhD

Session 9: Ethical & Regulatory Oversight Considerations
Stephanie Morain, PhD, MPH

Session 10: Writing a Compelling Grant Application
Beda Jean-Francois, PhD

Session 11: ePCTs in Context: Small Group Work Followed by Panel Discussion with NIH Collaboratory Trials PIs

Moderator: Emily O'Brien, PhD

*This session includes case study presentations.

Concluding Remarks

Emily O'Brien, PhD

Case Studies

Examples from NIH Collaboratory Trials

November 28, 2023: Workshop Summary Now Available From ‘Getting the Right Evidence to Decision-Makers Faster’

The workshop summary is now available from the NIH Pragmatic Trials Collaboratory’s recent workshop, “Getting the Right Evidence to Decision-Makers Faster.” The 2-day workshop explored the critical cycle of evidence generation by researchers to decision-making by healthcare system leaders to implement the findings of pragmatic clinical trials conducted within healthcare systems.

The workshop included 4 panels:

  • Panel 1: How Have Health Systems Made Decisions Based on Evidence Collected in PCTs?

    • Panelists: Devon Check, Vincent Mor, Lynn DeBar, Kathryn Glassberg, Douglas Zatzick, Eileen Bulger, Susan Huang, Kenneth Sands, Edward Septimus; Moderator: Gregory Simon
  • Panel 2: How Do We Generate the Right Evidence to Support Decision-Makers?

    • Panelists: Kenneth Sands, Eileen Bulger, Edward Septimus, Amy Kilbourne, Rosa Gonzalez-Guarda, Patrick Heagerty; Moderator: Hayden Bosworth
  • Panel 3: Learning Faster

    • Panelists: Gloria Coronado, Natalia Morone, Corita Grudzen, Kevin Chan, Pearl O’Rourke, Cheryl Boyce, Andrea Cook; Moderator: Kevin Weinfurt
  • Panel 4: Potential Structures and Incentives for Faster Learning

    • Panelists: David Chambers, Wynne Norton, Tisha Wiley, Kenneth Sands, Edward Septimus, Gloria Coronado, Natalia Morone, Corita Grudzen; Moderator: Richard Platt

Access the complete workshop materials, including slides and videocast recordings, as well as the keynote presentation by Andrew Bindman, executive vice president and chief medical officer for Kaiser Permanente.

Driving Tomorrow’s Outcomes Through Clinical Research in Real-World Settings: Essentials of Embedded Pragmatic Clinical Trials: AcademyHealth (June 2023)

June 23-24, 2023: This workshop introduced concepts in the design, conduct, and implementation of embedded pragmatic clinical trials, with a focus on methods relevant to health services researchers, while providing firsthand experiences and case studies from the NIH Pragmatic Trials Collaboratory Trials.

Agenda and Slides

June 23, 2023

Welcome and Opening Remarks

Kevin Weinfurt, PhD

Session 1: What are Embedded Pragmatic Clinical Trials (ePCTs)?
Wendy Weber, ND, PhD, MPH

Session 2: Engaging Stakeholders & Aligning with Health System Partners
Emily O’Brien, PhD

Session 3: Objectives and Trial Design: An Overview of Hybrid Designs
Hayden Bosworth, PhD

Session 4: Measuring Outcomes
Emily O’Brien, PhD

Session 5: ePCT Design
Patrick Heagerty, PhD

Session 6: ePCT Analysis
Patrick Heagerty, PhD

Session 7ePCTs in Context: Small Group Work Followed by Panel Discussion with NIH Collaboratory Trial PIs
Moderator:  Kevin Weinfurt, PhD

Panel:

  • Margaret Kuklinski, PhD
  • Angelo Volandes, MD
  • Michael Parchman, MD

Closing Remarks

Kevin Weinfurt, PhD

Getting the Right Evidence to Decision-Makers Faster: Insights From the NIH Pragmatic Trials Collaboratory: NIH-Hosted Workshop (June 2023)

June 20-21, 2023: This two-day workshop explores the critical cycle of evidence generation to decision by health system leaders to implement the findings of pragmatic clinical trials (PCTs) conducted within health care systems.

Agenda and Slides

Day 1: June 20

Welcome and Introduction

Wendy Weber, ND, PhD, MPH

Keynote: Learning Health Systems: Reflections from the C-Suite

Andrew B. Bindman, MD

Session 1: How Have Health Systems Made Decisions Based on Evidence Collected in PCTs?

Review of Collaboratory Experience
Devon Check, PhD

Moderator: Gregory Simon, MD, MPH

Panel:

Session 2: How Do We Generate the Right Evidence to Support Decision-Makers?

Moderator: Hayden Bosworth, PhD

Panel:

Summary and Concluding Remarks

Wendy Weber, ND, PhD, MPH
Richard Platt, MD, MSc
Gregory Simon, MD, MPH
Hayden Bosworth, PhD

 

Day 2: June 21

Welcome and Introduction

Wendy Weber, ND, PhD, MPH

Session 3: Learning Faster

Moderator/Speaker: Kevin Weinfurt, PhD

Panel:

Session 4: Potential Structures and Incentives for Faster Learning

Moderator: Richard Platt, MD, MSc

Panel:

  • David Chambers, DPhil
  • Wynne Norton, PhD
  • Tisha Wiley, MD

Respondents:

  • Kenneth Sands, MD
  • Edward Septimus, MD
  • Gloria Coronado, PhD
  • Natalia Morone, MD, MS
  • Corita Grudzen, MD, MSHS

Summary and Concluding Remarks

Wendy Weber, MD, PhD, MPH
Kevin Weinfurt, PhD
Richard Platt, MD, MSc

 

Changing Trials for Changing Times: Essentials of Embedded Pragmatic Clinical Trials Workshop: Health Care Systems Research Network Conference (February 2023)

February 20, 2023: The NIH Pragmatic Trials Collaboratory hosted a Pre-Conference Workshop at the Health Care Systems Research Network 2023 Annual Conference. This training workshop introduces concepts in the design, conduct, and implementation of embedded pragmatic clinical trials (ePCTs) and provides firsthand ePCT experiences and case studies from the NIH Pragmatic Trials Collaboratory Trials.

Agenda and Slides

Morning Sessions

Welcome and Introduction

Wendy Weber, ND, PhD, MPH

Session 1: What are Embedded Pragmatic Clinical Trials (ePCTs)?
Wendy Weber, ND, PhD, MPH

Session 2: Engaging Stakeholders & Aligning with Health System Partners
Emily O’Brien, PhD

Session 3: Objectives and Trial Design: An Overview of Hybrid Designs
Patrick Heagerty, PhD

Session 4: Measuring Outcomes
Emily O’Brien, PhD

Session 5: ePCT Design and Analysis
Patrick Heagerty, PhD

Session 6ePCTs in Context: Panel Discussion with NIH Collaboratory Trial PIs
Moderator: Kevin Weinfurt, PhD

*This session includes the Case Study Presentations.

 

Afternoon Sessions

Session 7: Pilot & Feasibility Testing
Wendy Weber, ND, PhD, MPH

Session 8: Ethical & Regulatory Oversight Considerations
Stephanie Morain, PhD, MPH

Session 9: Writing a Compelling Grant Application
Beda Jean-Francois, PhD

Session 10: ePCTs in Context: Panel Discussion with NIH Collaboratory Trial PIs
Moderator: Kevin Weinfurt, PhD

*This session includes the Case Study Presentations.

Session 11: Next Steps
Kevin Weinfurt, PhD

 

Ethical & Regulatory Dimensions of Pragmatic Clinical Trials: Special Grand Rounds Series (September 2022-February 2023)

In 2022-2023, the NIH Pragmatic Trials Collaboratory used its popular PCT Grand Rounds platform to host a special workshop series to examine ethical and regulatory challenges in pragmatic clinical trials. The series included moderated panel discussions, in which the NIH Collaboratory's Ethics and Regulatory Core brought together bioethicists, investigators, and regulatory experts to share lessons learned, discuss empirical findings, and explore remaining uncertainties.

Grand Rounds Sessions

September 9, 2022, 1:00 pm ET: Keynote: Building an Academic Learning Health System: Why Is It So Hard?

Speaker: Steven Joffe, MD, MPH
Moderator: Stephanie Morain, PhD, MPH

October 14, 2022, 1:00 pm ET: Responding (or Not) to Signals of Potential Clinical Significance in Pragmatic Clinical Trials 

Speakers: Joseph Ali, JD; Leslie J. Crofford, MD; Tanya Matthews, PhD
Moderator: Stephanie Morain, PhD, MPH

November 11, 2022, 1:00 pm ET: Data Sharing and Pragmatic Clinical Trials: Law and Ethics Amidst a Changing Policy Landscape 

Speakers: Stephanie Morain, PhD, MPH; Kayte Spector-Bagdady, JD, MBioethics
Moderator: P. Pearl O’Rourke, MD

December 9, 2022, 1:00 pm ET: The Stepped-Wedge Cluster Randomized Trial: Friend or Foe?

Speakers: David Magnus, PhD; Monica Taljaard, PhD
Moderator: Stephanie Morain, PhD, MPH

January 13, 2023, 1:00 pm ET: Ethical Considerations When Vulnerable Populations Are Subjects in Pragmatic Trials

Speaker: Emily A. Largent, JD, PhD, RN
Moderator: Stephanie Morain, PhD, MPH

February 10, 2023, 1:00 pm ET: Informing and Consenting: What Are the Goals?

Speakers: P. Pearl O’Rourke, MD; David S. Wendler, PhD, MA; Miguel Vazquez, MD; P. Michael Ho, MD, PhD
Moderator: Stephanie Morain, PhD, MPH

 

Dissemination & Implementation Research Methods and Embedded Pragmatic Trials: Strategies for Designing Studies That Inform Care for Diverse Populations: AcademyHealth Conference on the Science of Dissemination and Implementation in Health (December 2022)

December 11, 2022: The NIH Pragmatic Trials Collaboratory hosted a Pre-Conference Workshop at the 15th AcademyHealth Annual Conference on the Science of Dissemination and Implementation in Health. This training workshop introduces concepts in the design, conduct, and implementation of embedded pragmatic clinical trials (ePCTs), and provides firsthand ePCT experiences and case studies from the NIH Pragmatic Trials Collaboratory Trials.

Agenda and Slides

Morning Sessions

Welcome and Introduction

Wendy Weber, ND, PhD, MPH

Session 1: What are Embedded Pragmatic Clinical Trials (ePCTs)?
Wendy Weber, ND, PhD, MPH

Session 2: Engaging Stakeholders & Aligning with Health System Partners
Emily O’Brien, PhD

Session 3: Objectives and Trial Design: An Overview of Hybrid Designs
Emily O’Brien, PhD

Session 4: Measuring Outcomes
Emily O’Brien, PhD

Session 5: ePCT Design
David Murray, PhD

Session 6: ePCT Analysis
David Murray, PhD

Session 7ePCTs in Context: Panel Discussion with NIH Collaboratory Trial PIs
Moderator:  Emily O’Brien, PhD

*This session includes the Case Study Presentations.

 

Afternoon Sessions

Session 8: Pilot & Feasibility Testing
Wendy Weber, ND, PhD, MPH

Session 9: Ethical & Regulatory Oversight Considerations
Stephanie Morain, PhD, MPH

Session 10: ePCTs in Context: Panel Discussion with NIH Collaboratory Trial PIs
Moderator: Wendy Weber, ND, PhD, MPH

*This session includes the Case Study Presentations.

Session 11: Assembling an ePCT Team & Writing a Grant Application
Beda Jean-Francois, PhD

Session 12: Next Steps
Wendy Weber, ND, PhD, MPH

 

Critical Questions for Pragmatic Clinical Trialists: Insights From the NIH Pragmatic Trials Collaboratory’s First Decade: NIH-Hosted Workshop (June 2022)

June 15-16, 2022: This workshop is an opportunity to glean the most salient lessons learned from an experienced group of pragmatic trial investigators who will openly share the challenges they have encountered, solutions they have developed, and thoughts for the future. 

Agenda and Slides

Day 1: June 15, 2022

Welcome and Introduction

Wendy Weber, ND, PhD, MPH
Helene Langevin, MD

Keynote: Pragmatic Clinical Trials: The Role of Nursing Science

Shannon N. Zenk, PhD, MPH, RN, FAAN

Session 1: Can We Squeeze More Value Out of Electronic Health Records?

Moderator: Keith Marsolo, PhD

Panel:

  • ICD-Pieces™: Improving Chronic Disease Management with Pieces (Miguel Vazquez, MD)
  • LIRE: Lumbar Imaging with Reporting of Epidemiology (Jeffrey (Jerry) Jarvik, MD, MPH)
  • NOHARM: Nonpharmacologic Options in Postoperative Hospital-based and Rehabilitation Pain Management (Andrea Cheville, MD)
  • SPOT: Suicide Prevention Outreach Trial (Greg Simon, MD, MPH)

Session 2: Should We Be Considering a Different Study Design?

Moderator: Patrick Heagerty, PhD

Panel:

Summary and Concluding Remarks

Lesley Curtis, PhD
Wendy Weber, ND, PhD, MPH
Keith Marsolo, PhD
Patrick Heagerty, PhD

Day 2: June 16, 2022

Welcome and Introduction

Wendy Weber, ND, PhD, MPH
Helene Langevin, MD

Session 3: What Is the Most Effective Use of the 1-Year Planning Phase?

Moderator: Wendy Weber, ND, PhD, MPH

Panel:

Session 4: How Do We Implement Interventions in Complex Health Care Systems?

Moderator: Steven George, PT, PhD

Panel:

  • STOP CRC: Strategies and Opportunities to Stop Colorectal Cancer in Priority Populations (Gloria Coronado, PhD)
  • ICD-Pieces™: Improving Chronic Disease Management with Pieces (Miguel Vazquez, MD)
  • GGC4H: Guiding Good Choices for Health: Testing Feasibility and Effectiveness of Universal Parent-Focused Prevention in Three Healthcare Systems (Margaret Kuklinski, PhD)
  • GRACE: Hybrid Effectiveness-Implementation Trial of Guided Relaxation and Acupuncture for Chronic Sickle Cell Disease Pain (Miriam Ezenwa, PhD, MSN, BSN, RN, FAAN)

Summary and Concluding Remarks

Lesley Curtis, PhD
Wendy Weber, ND, PhD, MPH
Steven George, PT, PhD

 

Essentials of Embedded Pragmatic Clinical Trials Workshop: Health Care Systems Research Network Annual Conference (April 2022)

April 11, 2022: The NIH Pragmatic Trials Collaboratory hosted a Pre-Conference Workshop at the 2022 Health Care Systems Research Network (HCSRN) Annual Conference.  This training workshop introduces concepts in the design, conduct, and implementation of embedded pragmatic clinical trials (ePCTs), and provides firsthand ePCT experiences and case studies from the NIH Pragmatic Trials Collaboratory Trials.

Agenda and Slides

Welcome and Introduction

Kevin Weinfurt, PhD

Session 1: What Are Embedded PCTs (ePCTs)?
Kevin Weinfurt, PhD

Session 2: Objectives and Trial Design: An Overview of Hybrid Design
Devon Check, PhD

Session 3: Measuring Outcomes
Devon Check, PhD

Session 4: ePCT Experimental Design & Analysis
Patrick Heagerty, PhD

Session 5: Pilot & Feasibility Testing
Miguel Vazquez, MD

Session 6: Ethical & Regulatory Oversight Considerations
Kevin Weinfurt, PhD

Session 7: Writing a Compelling Grant Application
Michael Ho, MD, PhD

Session 8: ePCTs in Context: Panel Discussion
Moderator: Kevin Weinfurt, PhD

*This presentation contains case studies.

Session 9: Next Steps
Kevin Weinfurt, PhD

 

Convergence of Dissemination & Implementation Research Methods and Embedded Pragmatic Trials: Examples from the NIH Health Care Systems Research Collaboratory and NIA IMPACT ADRD Collaboratory (Virtual, December 2021)

December 13, 2021: The NIH Pragmatic Trials Collaboratory and the NIA IMPACT Collaboratory co-hosted a Pre-Conference Workshop for Academy Health’s Annual Conference on the Science of Dissemination and Implementation in Health Conference (D&I).  This training workshop introduces concepts in the design, conduct, and implementation of embedded pragmatic clinical trials (ePCTs), with a particular focus on the intersection between ePCTs and implementation research methods. The workshop provides an introduction to the investigative opportunities for embedded health systems research, along with strategies for integrating implementation research methods in the design and conduct of ePCTs with the goal of providing real-world evidence necessary to inform both practice and policy. Firsthand ePCT experiences and case studies from the NIH Health Care Systems Research Collaboratory and the NIA IMPACT ADRD Collaboratory support and illustrate the topics presented.

Agenda and Slides

Welcome and Introduction

Wendy Weber, ND, PhD, MPH

Session 1: What Are Embedded Pragmatic Clinical Trials (ePCTs) and How Do They Intersect with D&I Research?

Wendy Weber, ND, PhD, MPH

Session 2: Stakeholder Engagement and Planning for D&I From the Beginning

Devon Check, PhD

Case Study Presentations
Moderator: Kevin Weinfurt, PhD

Session 3: Integrating D&I Into ePCT Study Designs & Analysis

An Overview of Hybrid Designs
Geoff Curran, PhD

Case Study Presentations
Moderator: Devon Check, PhD

Session 4: Additional Considerations When Conducting ePCTS

Ethical and Regulatory Considerations
Stephanie Morain, PhD

Pilot and Feasibility Testing
Wendy Weber, ND, PhD, MPH

A Framework for Achieving Health Equity in Pragmatic Trials
Jonathan Jackson, PhD

Case Study Presentations
Moderator: Vince Mor, PhD

Closing Remarks

David Chambers. PhD