Speaker
Ming Tai-Seale, PhD, MPH
Professor
Departments of Family Medicine and Medicine (Bioinformatics)
University of California San Diego School of Medicine
Director of UC San Diego Learning Health Systems Science Center
Keywords
Electronic Health Record; Artificial Intelligence; MyChart; Patient Messages; Large Language Models; Clinician Well-Being; Mental Health
Key Points
- Physician work is increasingly centered around the electronic health record (EHR). It consumes nearly 50% of scheduled clinic time. The volume of patient messages in MyChart increased significantly from 2020 to 2022, and remains much higher than pre-pandemic levels.
- Research published in Health Affairs and JAMA Network Open suggests that this influx of inbox messages is detrimental to physicians’ well-being. The emotional timbre of messages from patients plays a role, as well; in an analysis of EHR inbasket messages, the research team found messages from patients that contained expletives, vitriol and personal attacks.
- The research team sought to examine the association between generative AI (GenAI)-drafted replies for patient messages and physician time spent on answering messages. They were also looking at the quality of GenAI-drafted replies for messages dealing with mental heath concerns.
- The team created a prompt within the EHR that gave physicians the option to either use an AI-generated response as a starting point or to start with a blank reply. Messages eligible for responses drafted by GenAI included refills, results, paperwork, and general questions.
- The pilot study took place from June 16 to July 12, 2023, targeting primary care attending physicians at University of California San Diego. 52 physician volunteers received the intervention; the 70 physicians in the control arm did not.
- In the pilot study, clinicians who were given the option of a GenAI-drafted reply spent more time reading patient messages. There was no change in average reply time.
- When clinicians received messages dealing with mental health issues, replies drafted by more recent versions of GenAI had more utility than older versions.
- The physicians expressed that they valued the GenAI-drafted replies as a compassionate starting point for their communication. They noted areas for improvement, like a robotic tone, and emphasized the continued need for human oversight and intervention.
- The study team acknowledged potential risks when using large language models (LLMs) in mental health communication. These included a loss of human touch and empathy; overreliance and deskilling; and privacy and security risks.
- This is an ongoing effort. Next steps include using LLMs to facilitate analyses of qualitative data on electronic patient-clinician communication; triangulating qualitative and quantitative data in the EHR; and aiming for a more comprehensive understanding of mental health communication and how LLMs might improve its quality.
Discussion Themes
Anecdotally, the researchers have heard from physicians that ART technology – which Epic and Microsoft continue to refine – seems to have improved. But issues still remain, such as GenAI recommending patients see clinicians from external hospital systems.
When a modified GenAI-drafted reply was sent to a patient, a disclaimer was included: “Part of this message was generated automatically.” The research team felt that it was important to provide this transparency and disclose to patients when AI contributed to the messaging they received.
Health systems and professional organizations must develop standards advocating for equity in the implementation of and access to these tools.