March 3, 2020: Healthcare System Leaders Give Advice to Pragmatic Trial Investigators

In a new section of the Living Textbook on Building Partnerships to Ensure a Successful Trial, Kenneth Sands, the chief epidemiologist and chief patient safety officer at HCA, and Jonathan Perlin, the chief medical officer and president of clinical services at HCA, offer advice to investigators who wish to conduct pragmatic clinical trials.

“Healthcare systems have constrained bandwidth, and participating in learning activities involves significant costs and challenges: there are direct costs but also intangible costs, which can be substantial, such as personnel time, IT time, distraction of clinical staff, and the potential for supply chain issues.” — Advice from Healthcare System Leadership

The authors establish principles of partnership that include establishing and maintaining a durable partnership, selecting topics of mutual interest, and giving precedence to operational imperatives.

For more, see the article “Real-World Advice for Generating Real-World Evidence” in NEJM Catalyst.

May 3, 2019: Effect of Financial Bonus Sizes, Loss Aversion, and Increased Social Pressure on Physician Pay-for-Performance: A Randomized Trial and Cohort Study (Amol Navathe, MD, PhD)

Speaker

Amol S. Navathe, MD, PhD
Assistant Professor of Medicine and Health Policy
University of Pennsylvania

Topic

Effect of Financial Bonus Sizes, Loss Aversion, and Increased Social Pressure on Physician Pay-for-Performance: A Randomized Trial and Cohort Study

Keywords

Behavioral economics; Performance incentives; Evidence-based quality-of-care measures; Primary care quality; Pay for performance; Value-based medicine

Key Points

  • Pay-for-performance (P4P) programs are increasingly being used by health insurers and healthcare systems to incentivize physicians to practice higher value medicine, yet the evidence for P4P to affect quality and value of care remains mixed.
  • Behavioral economic principles in this study included increased social pressure and loss aversion added to larger bonus sizes to evaluate whether the intervention would lead to higher achievement of evidence-based quality measures.

Discussion Themes

Study findings included that, while a larger bonus size was associated with significantly improved quality for chronic care patients relative to a propensity-matched comparison group, adding increased social pressure and the opportunity for loss aversion did not lead to further quality improvement.

Attrition during the trial contributed some variability to the analysis.

Read more about pay for performance in healthcare in JAMA Network Open (Navathe et al, 2019) and NEJM Catalyst (2018).

Tags

#behavioraleconomics, #pctGR, @Collaboratory1

April 18, 2019: New Commentary Highlights Value of Pragmatic Trials for Learning Health Systems

In an eGEMs commentary published this month, Leah Tuzzio and Dr. Eric Larson of the NIH Collaboratory’s Health Care Systems Interactions Core discuss the value and impact of embedded pragmatic clinical trials for learning health systems.

Pragmatic trials embedded in healthcare systems are designed to align with the care delivery goals of the health system to produce better health outcomes. The commentary highlights the NIH Collaboratory’s pragmatic trials as “the best-case examples to learn about the challenges of conducting research and of dissemination, implementation and sustainability of research results in real-world settings.”

Designing and implementing an embedded pragmatic clinical trial “requires a bidirectional flow of information and cooperative problem solving between investigator teams and clinical teams,” an important feature of learning health systems. In implementing the trial, the clinical and research teams not only generate useful data, but also demonstrate how the trial results can be incorporated into evidence-based clinical practice.

The commentary is part of a special collection of eGEMs articles commemorating 25 years of the Health Care Systems Research Network.

eGEMs is AcademyHealth’s peer-reviewed, open-access journal for electronic health data and methods. At AcademyHealth’s 2019 Annual Research Meeting in Washington, DC, the NIH Collaboratory will offer a full-day pre-conference seminar on the essentials of embedded pragmatic clinical trials. Registration for the seminar is open now.

June 14, 2018: NIH Collaboratory Members to Host Session on Embedded PCTs at AcademyHealth Annual Research Meeting

The upcoming AcademyHealth Annual Research Meeting, June 24-26, 2018, will feature a special session on Embedding Pragmatic Clinical Trials in Health Care Systems: Trials and Tribulations, Monday, June 25, at 5:15 pm, hosted by members of the Health Care Systems Interactions Core Working Group:

Chair: Cathy Meyers, MD, NIH/NICCH

Discussant: Eric B. Larson, MD, MPH, Kaiser Permanente Washington Health Research Institute

Speakers: Vince Mor, PhD, Brown University School of Public Health; Greg Simon, MD, MPH, Kaiser Permanente Washington Health Research Institute; and Lynn DeBar, PhD, MPH, Kaiser Permanente Washington Health Research Institute

Held at the Washington State Convention Center in Seattle, this large gathering of health services researchers and policy analysts will include workshops, poster and podium sessions, emerging issues panels, policy roundtables, and special topic sessions. Other NIH Collaboratory members planning to present at the meeting include Leah Tuzzio, MPH, Jerry Jarvik, MD, MPH, Miguel Vazquez, MD, Kathryn James, MPH, Gloria Coronado, PhD, and Beverly Green, MD, MPH.

 “Attendees will learn about the challenges of pilot testing, studying patient-reported outcomes, using existing data, and the multiple levels of implementation in dynamic systems.” – Leah Tuzzio, Kaiser Permanente Washington Health Research Institute

For details, visit the meeting event site at http://www.academyhealth.org/events/site/2018-annual-research-meeting.

STOP CRC Trial: Analytic Challenges and Pragmatic Solutions


Investigators from the STOP CRC pragmatic trial, an NIH Collaboratory Trial, have recently published an article in the journal eGEMs describing solutions to issues that arose in the trial’s implementation phase. STOP CRC tests a program to improve colorectal cancer screening rates in a collaborative network of Federally Qualified Health Centers by mailing fecal immunochemical testing (FIT) kits to screen-eligible patients at clinics in the intervention arm. Clinics in the control arm provided opportunistic colorectal-cancer screening to patients at clinic visits in Year 1 and implemented the intervention in Year 2. In this cluster-randomized trial, clinics are the unit of analysis, rather than individual patients, with the primary outcome being the proportion of screen-eligible patients at each clinic who complete a FIT.

The team dealt with various challenges that threatened the validity of their primary analysis, one of which related to potential contamination of the primary outcome due to the timing of the intervention rollout: for control participants, the Year 2 intervention actively overlapped with the Year 1 control measurements. The other challenge was due to a lack of synchronization between the measurement and accrual windows. To deal with these issues, the team had to slightly modify the study design in addition to developing a few sensitivity analyses to better estimate the true impact of the intervention.

“While the nature of the challenges we encountered are not unique to pragmatic trials, we believe they are likely to be more common in such trials due to both the types of designs commonly used in such studies and the challenges of implementing system-based interventions within freestanding health clinics.” (Vollmer et al. eGEMs 2015)

The Publish EDM Forum Community publishes eGEMs (generating evidence & methods to improve patient outcomes) and provides free and open access to this methods case study. Readers can access the article here.


New Lessons Learned Document Draws on Experiences of NIH Collaboratory Trials

The NIH Collaboratory’s Health Care Systems Interactions Core has published a document entitled Lessons Learned from the NIH Health Care Systems Research Collaboratory Trials. The Principal Investigators of each of the NIH Collaboratory Trials shared their trial-specific experience with the Core to develop the document, which presents problems and solutions for initiation and implementation of pragmatic clinical trials (PCTs). Lessons learned are divided into the following categories: build partnerships, define clinically important questions, assess feasibility, involve stakeholders in study design, consider institutional review board and regulatory issues, and assess potential issues with biostatistics and the analytic plan.

Other tools available from the Health Care Systems Interactions Core include a guidance document entitled Considerations for Training Front-Line Staff and Clinicians on Pragmatic Clinical Trial Procedures and an introduction to PCTs slide set.

Minimizing the Burden of Practical Research: Case Studies from the NIH Collaboratory


In a forthcoming article in Healthcare, Dr. Eric Larson and colleagues present practical advice based on case studies from the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Physician–scientists, health services researchers, and delivery system leaders provide insight from their experience launching a pragmatic clinical trial (PCT) as part of the Collaboratory.

The authors make 5 recommendations:

  • Establish a partnership from the get-go
  • Do a pilot project
  • Take advantage of existing hospital and health system infrastructure
  • Minimize the impact on clinical workflow
  • Remember that even high-priority research questions must be balanced with the systems’ greatest priority: providing good healthcare to patients.

The authors note that researchers need to be flexible and prepared to adjust the study design to the workflow and culture of the system.

Reference: Larson E, Tachibana C, Thompson E, et al. Trials without tribulations: Minimizing the burden of pragmatic research on healthcare systems. Healthcare. 2015; in press. doi:10.1016/j.hjdsi.2015.07.005

Watch Dr. Larson’s Grand Rounds Presentation from June 2013: Trials, Not Tribulations: Minimizing the Burden of Research on Health Care Systems