Medical devices

Grand Rounds December 6, 2024: Opportunities and Challenges in the Use of Large Language Models for Post-Marketing Surveillance of Medical Products (Michael E. Matheny, MD, MS, MPH)

Posted on by Sophia Ramirez

Speaker

Michael E. Matheny, MD, MS, MPH
Director, Center for Improving the Public’s Health Through Informatics
Professor of Biomedical Informatics, Biostatistics, and Medicine
Vanderbilt University Medical Center
Staff Scientist, Geriatrics Research Education and Clinical Care Service
Associate Director, VA ORD VINCI
Tennessee Valley Healthcare System VA

Keywords

Artificial Intelligence; Large Language Models; Surveillance; Medical Products

Key Points

  • Increasingly, leaders in many disciplines are finding new applications for Artificial Intelligence (AI). Within healthcare, this technology is being used to support clinical decision-making; imaging processing; drug discovery; clinical trials; and as Ambient and Autonomous AI.
  • Large Language Models (LLMs) are a subset of generative AI. Since 2012, LLMs have emerged as a promising new technology with rapid growth, evolution of capacity and reach, and many potential applications in healthcare and clinical research.
  • There is significant interest in using LLMs to assist with patient trial matching, clinical trial planning, and the development of trial protocols and consent documents.
  • Another key area that LLMs could provide support in is medical product safety surveillance, with potential applications in adverse event detection, probabilistic phenotyping, and information synthesis.
  • The post-marketing surveillance space utilizes an ecosystem of healthcare data, imaging, radiology reports, insurance, structured data, medical literature, and social media. These sources could be integrated to conduct LLM reasoning and extractions.
  • Key challenges in safe and effective use of LLMs for this purpose include the lack of evaluation for medical product surveillance, the complexities of prompt engineering, hallucination risk (i.e., false positives), and the fact that evolving models over time challenge stable performance estimates.

Discussion Themes

 A segment of the clinical workforce could be trained to be “super users,” partnering with development teams in order to make sure that these tools are working appropriately in a clinical environment.

There is substantial interest in using LLMs to support clinical decision-making. However, studies have shown that the quality of the AI output can influence the performance of the clinicians. Especially in high-risk clinical environments, any drift in those algorithms could result in adverse clinical outcomes. The life cycle approach to conceptualization, development, implementation, surveillance, and maintenance will be necessary to achieve and maintain performance.

October 12, 2018: MDEpiNet RAPID and SPEED Projects: Leveraging Real World Evidence to Get Better, Faster, Cheaper Medical Devices for Physicians and Patients (Renee Mitchell, MT, CLS, Terrie Reed, MSIE, Roseann White, MA)

Posted on by Gina Uhlenbrauck

Speakers

Renee Mitchell, MT(ASCP), CLS(NCA)
Regulatory Affairs
Boston Scientific Corporation, Inc.

Terrie Reed, MSIE
Senior Advisor for UDI Adoption
U.S. Food and Drug Administration (FDA)

Roseann White, MA
Director of Innovative Clinical Trial Statistics
Duke Clinical Research Institute

Topic

MDEpiNet RAPID and SPEED Projects: Leveraging Real World Evidence to Get Better, Faster, Cheaper Medical Devices for Physicians and Patients

Keywords

Medical devices; Real-world evidence; Medical Device Epidemiology Network; MDEpiNet; Unique device identifier; UDI

Key Points

  • In partnership, clinicians, device developers, and FDA can benefit from the use of real-world evidence on medical devices:
    •  Clinicians can contribute to the generation of real-world evidence.
    •  Device manufacturers can use real-world evidence to evaluate and release new devices and expand indications.
    •  Regulatory bodies can increase the use of patient-level data for device approval.
  • Unique device identifiers (UDIs) make it possible to follow medical devices longitudinally, advancing the quality of real-world evidence and allowing more sophisticated analyses.

Discussion Themes

The vision for the future is that registries will transform into big data solutions using multiple sources and will be more robustly integrated with electronic health records (EHRs). Both EHRs and registries will play a role.

More organizations as partners brings greater diversity, advancing better data and results. When stakeholders work together, learning curves can be accelerated toward a transformational approach to real-world evidence.

Tags

#MedicalDevices, #pctGR, @PCTGrandRounds, @Collaboratory1, @MDEpiNet_ppp

Task Force Releases Recommendations for National Medical Device Evaluation System

Posted on by Gina Uhlenbrauck

A new report (PDF) containing recommendations for the creation of a national registry system for evaluating and monitoring medical devices has been released for public comment today. The report, a joint project of the Medical Device Registry Task Force and cover_19aug2015 the Medical Device Epidemiology Network (MDEpiNet), is available on boh the US Food and Drug Administration (FDA) website and on  the MDEpiNet website.

The report reflects the results of a year-long effort, prompted by the FDA’s Center for Devices and Radiological Health (CDER), that  is focused on fostering a national system for monitoring the use of medical devices in the “real-world” setting of patient care, once the devices have been approved for the market (known as “postmarket surveillance”).

The term “medical devices” encompasses a wide range of technologies, including implantable pacemakers, cardiovascular stents, robotic surgical devices, and artificial joint replacements, among many others. At present, information about the use of these devices in routine care settings, including safety issues reported by doctors and patients, is collected in a variety of registries and health record systems. A  networked national system, such as the one described in the task force report, would be able to unite and build upon both existing and novel data resources, thereby improving safety monitoring and accelerating the development of new devices:

“Task Force recommendations for [Coordinated Registry Network] CRN architecture, and thus for the National System, center on leveraging existing, self sustaining electronic resources, such as device registries, electronic health records, administrative data and even social media and personal mobile device sources.”

The Task Force Report offers recommendation in several key areas, including:

  • Establishing a national dialog about medical device evaluation that includes all stakeholders;
  • Leveraging existing efforts in the arena of device registries and electronic data systems;
  • Describing the desired characteristics of a national Coordinated Registry Network (CRN) for medical devices;
  • Outlining priorities for developing and refining medical devices in multiple therapeutic areas;
  • Identifying and improving methods for analyzing data on medical devices; and
  • Addressing network governance and issues related to patient privacy and informed consent.

Each of these key areas also features suggested pilot projects designed to inform ongoing efforts.

A related perspective article summarizing the National Registry System project has also been published online in the Journal of the American Medical Association.

Related Links