AI

Grand Rounds March 21, 2025: Generative Artificial Intelligence in Clinical Trials: A Driver of Efficiency and Democratization of Care (Alexander J. “AJ” Blood, MD, MSc)

Posted on by Sophia Ramirez

Speaker

Alexander J. “AJ” Blood, MD, MSc
Associate Director, Accelerator for Clinical Transformation Research Group
Instructor of Medicine at Harvard Medical School
Cardiologist and Intensivist
Brigham and Women’s Hospital

Keywords

Artificial Intelligence; Cost; Large Language Models; Enrollment; Eligibility; Recruitment

Key Points

  • The Accelerator for Clinical Transformation (ACT) is a research group that seeks to use emerging technology to try and expand access to healthcare and improve quality and quantity of healthcare delivery. They focus on team-based models and scalable applications.
  • It’s becoming more expensive and time-consuming to move a drug from the clinical trial stage to approval. Patient recruitment is the leading driver of costs in clinical trials, and 55% of trials that fail to complete cite low accrual rate as the reason for study termination. There’s pressure from industry to conduct clinical trials in a way that is faster, cheaper, and better for both the patients and the research environment.
  • ACT conducted a pilot study in which they embedded a Large Language Model (LLM) tool called RECTIFIER into an active clinical trial of patients with heart failure. RECTIFIER is an AI-powered, comprehensive software application able to ask and answer questions about unstructured clinical data. In a pilot study, RECTIFIER determined patient eligibility with higher accuracy and specificity than study staff, indicating its potential to streamline screening.
  • LLMs are the engines that power the software. There are two key challenges that need to be taken into consideration to use these tools effectively: 1) there’s a content window – a limit to the amount of Electronic Health Record (EHR) data you can pull in; and 2) Using LLMs is expensive.
  • Following up on the pilot study results, ACT conducted a prospective randomized controlled trial: The Manual Versus AI-Assisted Clinical Trial Screening Using LLMs (MAPS-LLM) trial. MAPS-LLM compared two methods for analyzing a randomized pool of potentially eligible participants: manual review by study staff, and RECITIFIER-augmented review by study staff. Their primary endpoint was eligibility determination.
  • They found that AI-assisted patient screening using the RECTIFIER system significantly improved eligibility determination and enrollment compared with manual screening in a heart failure clinical trial.
  • ACT concluded that implementing AI-assisted tools like RECTIFIER can enhance clinical trial efficiency, reduce resource utilization, and promote equitable recruitment, potentially leading to faster trial completion and earlier patient access to novel therapies. Generative AI is likely to play a significant role in the future of clinical trials.

Discussion Themes

Study staff in the MAPS-LLM intervention arm were able to direct more time and effort towards contacting patients and managing patients with the time they would have spent reviewing charts and manually screening the EHR.

The rate of eligibility between the two arms was equivalent; the difference was, the AI-augmented group was able to assess twice as many potentially eligible patients.

While this tool can do a lot of analytical work, a human element will be essential to utilizing it effectively and to bringing “human intelligence” to participant enrollment.

The ACT team has started to pilot this technology in other disease areas, including cardiology more broadly, endocrinology, oncology, and gastroenterology.

Grand Rounds February 21, 2025: Texting for Behavior Change: Lessons Learned Across 2 Interventions to Improve Chronic Care Management (Michael Ho, MD, PhD; Sheana Bull, PhD)

Posted on by Sophia Ramirez

Speakers

Michael Ho, MD, PhD
Kaiser Permanente Colorado

Sheana Bull, PhD
University of Colorado School of Public Health

Keywords

Text Messaging; Artificial Intelligence; Chatbots; Health Behaviors

Key Points

  • Ample evidence now exists demonstrating the benefit of using text messaging in support of health behavior and access to care. It’s ubiquitous, increasing reach; theory in message design is impactful; and it can improve adherence to medical appointments and health behaviors.
  • Two NIH Collaboratory Trials, Nudge and Chat 4 Heart Health (C4HH), test the effectiveness of text messaging interventions to support behavior change. Nudge randomized patients to receive usual care, generic texts, behavioral texts, or behavioral texts plus chatbot messages. Their primary outcome was medication adherence.
  • C4HH, the subsequent trial, is randomizing patients to receive a generic text message curriculum; an AI chatbot messaging curriculum; or AI chatbot messages plus proactive pharmacist support. Their primary outcome is cardiovascular risk factors, as measured by the American Heart Association’s “Life’s Essential 8” adherence.
  • Nudge used an opt-out consent approach where CC4H used an opt-in consent approach. In the former, the research team noted, patients who identified as Black, Hispanic, and primary Spanish speakers were more likely to remain in the study. An opt-out approach in the appropriate context may be a way to diversify clinical trial populations and improve external validity of results.
  • The use of AI chatbots allows users to generate questions in their own words and the system to retrieve a response from a closed, curated library.
  • Message engagement is key to text messaging interventions. Participants in the Nudge study who were randomized to optimized texts had more questions. Questions were related to medications, refill logistics, and costs. The study team hypothesizes that the optimized texts may have led to greater patient engagement, and therefore more questions about their medications.
  • Over 12 months, the Nudge study found no significant difference in the rates of prescription refills, between the 3 intervention arms and usual care. CC4H is ongoing, and will send a higher volume of messages in an effort to engage patients and change patient behavior.
  • So far, the top 5 topics in messages initiated by C4HH participants have been healthy eating, physical activity, managing cholesterol, quitting smoking, and medication management.

Discussion Themes

The study team had to be very careful to ensure that patient health data, including cell phone numbers and the messages sent, were encrypted. Vendors and phone carriers were not able to access this data and it was not stored on their servers.

One of the challenges they encountered was that their systems weren’t integrated into the health care organizations’ pharmacies or electronic health records. The integration piece will be key to any future sustainability.

As technology evolves significantly over the course of, say, a 5-year study, developing the skillset to utilize interactive interventions or a SMART design could be helpful for investigators interested in conducting research in this area.

November 12, 2024: Duke-Margolis and Duke Health to Co-Host Webinar on AI Governance in Health Systems

Posted on by Sophia Ramirez

On November 18, 2024 at 2 pm eastern, the Duke Margolis Institute for Health Policy and Duke Health will co-host “From Principles to Practice: Exploring AI Governance in Health Systems.” In this public webinar, Duke-Margolis and Duke Health will discuss their newly released white paper on how health systems are navigating the role of AI governance.

The webinar will begin with an overview presentation of key takeaways from the white paper, followed by a fireside chat where experts will discuss the benefits of governance and lessons learned while building their own AI governance processes. After the fireside chat, there will be a panel discussion on methods and supports to facilitate the democratization of AI governance so more health organizations can safely and responsibly use these novel tools.

Registration is required for participation, but there is no cost to attend. Continuing education credits are available for several disciplines for participants who are affiliated with VA. Contact margolisevents@duke.edu with any questions.

Learn more and register today.

Grand Rounds June 28, 2024: Using ChatGPT to Facilitate Truly Informed Medical Consent (Fatima N. Mirza, MD, MPH)

Posted on by Sophia Ramirez

Speaker

Fatima N. Mirza, MD, MPH
Chief Resident
Department of Dermatology
Warren Alpert Medical School of Brown University

Keywords

Artificial Intelligence; ChatGPT; Informed Consent

Key Points

  • Artificial Intelligence (AI), when implemented thoughtfully in clinical settings, can lead to meaningful results for patients.
  • Medicine has long fallen short of the ideal of informed consent, in part due to the limited readability of informed consent forms; as of 2020, 54% of Americans were estimated to read below the sixth-grade reading level. This has implications for patient understanding and quality of care.
  • Using ChatGPT-4, the research team took LifeSpan Healthcare System’s (LHS) surgical consent form from a 12.6 Flesch-Kincaid reading level to a 6.7. After hospital leadership reviewed the revised form and the research team had addressed concerns around biases, approvals, and future needs, the revised form was deployed across LHS.
  • This real-world implementation demonstrated the potential for AI to make meaningful improvements to patient care and communication. As a proof-of-concept, it sparked the interest of other health systems.
  • To investigate the clinical implications of consent form readability, the research team analyzed 798 federally funded clinical trials providing accessible informed consent forms. A 16% increase in the dropout rate was associated with each additional Flesch-Kincaid grade-level increase in the language.
  • The observed association between consent form complexity and participant dropout rate could be attributed to misaligned expectations; erosion of trust; participant surprise and dissatisfaction; and reduced engagement. This highlights the importance of clear, accessible communication throughout the entire trial process, not just enrollment.
  • Improved readability of consent forms is crucial for the design and implementation of clinical trials, potentially leading to more inclusive, efficient, and impactful clinical research.

Discussion Themes

Medical malpractice attorneys were part of the cohort that reviewed the simplified consent forms. They shared that when these cases go to court, a jury will often review the informed consent document. In these cases, it can more protective to have documentation that people can understand.

There is the potential for tailored chatbots that could personalize the consent process for patients, but expert oversight would be crucial. AI models can “hallucinate,” saying something incorrect with absolute certainty. Dr. Mirza indicated that the field isn’t ready for those bespoke consents; centralized documents, as least for the time being, are the way to go.

AI is going to be integrated into our healthcare system. It’s important for clinicians, researchers, and people who really care about patients, as well as the patients themselves, to have a seat at the table discussing these early models.