In a new episode of the Rethinking Clinical Trials Podcast, Majid Afshar and Mary Ryan Baumann of the University of Wisconsin-Madison expanded on key takeaways from their recent Grand Rounds presentation, “A Pragmatic Randomized Controlled Trial of Ambient Artificial Intelligence (AI) to Improve Health Practitioner Well-Being.”
Listen to the podcast. For alerts about new episodes, subscribe for free on Spotify, Amazon Music, Apple Podcasts, or SoundCloud.
The need to document patient visits in the electronic health record is a driver of clinician burnout. The research team sought to assess whether the use of an ambient AI scribe could improve clinician well-being by automating the documentation process. In the podcast, Afshar and Ryan Baumann reflected on some of the forces that guided their trial design.
“The technology itself is in silo, right? The speech-to-text and text-to-speech and [large language models] are very good,” said Afshar. “I think the challenge is actually adapting it to the human use case… We spent a lot of time with our human factors engineering implementation scientists to make sure that the training and onboarding and the use of the tool was appropriate for given end users.”
According to Afshar, this predeployment step led to a fidelity rate of over of 70%.
Once the research team began collecting data, the combination of new digital technologies and a pragmatic setting posed its own set of challenges.
“The pure amount of information that’s coming from each participant is astounding,” said Baumann. “It’s not on a scale that you can look at each individual data point and necessarily understand: Is this an outlier? Did something weird happen on this one day? What are the other elements and factors within a health system that’s still operating as it normally does?”
Ultimately, the research team found a 20% reduction in burnout and a trend towards improvement in professional fulfillment. UW Health scaled the project from 66 to over 600 licenses.
Afshar is an associate professor and director of learning health systems in the Departments of Medicine and Biostatistics & Medical Informatics. Baumann is an assistant professor in the Departments of Population Health Sciences and Biostatistics & Medical Informatics.