Speakers
Sreek Vemulapalli, MD
Associate Professor of Medicine with Tenure
Director, CV Imaging, DUHS
Duke Clinical Research Institute
Duke-Margolis Institute for Health Policy
Ricardo Henao, PhD
Associate Professor
Department of Biostatistics & Bioinformatics
Department of Electrical and Computer Engineering
Department of Surgery
Associate Director of Clinical Trials AI, Duke Clinical Research Institute (DCRI)
Duke University
Keywords
Adjudication; AI; Clinical Event; Clinical Event Committees; Scalability
Key Points
- The promise of clinical endpoint or event adjudication is standardization and the improvement of endpoint credibility. However, a human-centered review model utilizing Clinical Endpoint Committees (CECs) represents a major scalability constraint due to cost pressure, operational drag, and between-group variability.
- AI could reduce the burden on CECs by separating confident events from cases requiring human review. Previous trials testing this concept have derived separate models for each event type and have no demonstrated ability to adapt to new definitions or endpoints across trials. The research team sought to develop and train an AI algorithm to perform clinical endpoint adjudication that can extend to new endpoint and trials. They then compared that algorithm’s performance to Chat GPT 4.0 and hybrid (AI + human) adjudication.
- The research team found that a single AI algorithm trained on a single trial can, after brief adaptation, identify the same events with a new definition and identify new events. This was validated in the EUCLID trial, when the algorithm alone or as a hybrid strategy reproduced the EUCLID primary endpoint and effect size. While further prospective studies are needed, the potential impact of AI-enabled automated adjudication of outcomes on developing evidence is substantial.
Discussion Themes
The role of clinical adjudicators could shift toward auditing the AI, similar to the evolution of EKG reading, where experts focus on the 10% of cases where the algorithm is most likely to be wrong or uncertain.
The primary barriers to implementing AI systems that comb through electronic health records for trial data are governance, data security, and access rights, rather than a lack of technological capability.
While demonstrated here in cardiovascular trials, this methodology is theoretically applicable to other complex domains. To gain regulatory acceptance, the presenters suggested that prospective validation across multiple clinical research groups and a wide variety of endpoints will be necessary.