In a study that used large language models (LLMs) to adjudicate documented goal-concordant care from longitudinal electronic health record (EHR) data, LLM-based ratings and rationales were broadly consistent with human adjudication. The study suggests that LLM-based tools show promise as complementary adjudicators of such records in research that would otherwise rely on labor-intensive manual chart review.
The research was published online ahead of print in the American Journal of Hospice and Palliative Medicine.
The proof-of-concept study used data from the ACP PEACE trial and one other pragmatic clinical trial of advance care planning interventions. ACP PEACE, an NIH Collaboratory Trial, evaluated the delivery of a video decision aid to patients together with goals-of-care communication skills training to oncology clinicians.
In the study, LLM-based ratings compared favorably with human ratings in a highly structured review of longitudinal records for goal-concordant care. However, the study also highlighted the subjectivity of such assessments. The authors stress that LLM-based tools should be viewed as complements to human review, rather than replacements, especially in complex and ambiguous cases.
ACP PEACE was supported within the NIH Pragmatic Trials Collaboratory by a grant from the National Institute on Aging. Learn more about ACP PEACE.
In 2025, NIH Pragmatic Trials Collaboratory investigators published new study designs and trial results, shared insights from program leadership, and developed innovative methods in the design, conduct, implementation, and dissemination of pragmatic clinical trials. Their work included perspectives from the Coordinating Center, best practices from the 
In a new episode of our Rethinking Clinical Trials Podcast, Drs. Angelo Volandes and James Tulsky speak with host Dr. Adrian Hernandez about the results of the ACP PEACE study.
Even in minimal-risk studies that do not use the standard consent process, there may be value in informing participants about the research. Such notifications should be considered the default for clinical trials conducted under a waiver of informed consent, argue the authors of a new report from the NIH Pragmatic Trials Collaboratory.
IMPACt-LBP, currently in the 1-year planning phase, will evaluate implementation of the American College of Physicians guideline for low back pain, which involves multidisciplinary collaborative care that includes doctors of chiropractic and physical therapists. The study will measure the effects of first-contact patient referral to these clinicians on physical function, pain, opioid prescriptions, and other patient-level outcomes. The study is administered by the National Center for Complementary and Integrative Health with additional support from the National Institute of Arthritis and Musculoskeletal and Skin Diseases and the National Institute of Child Health and Human Development. The project is led by Drs. Christine Goertz, Adam Goode, and Hrishikesh Chakraborty of Duke University and Dr. Jon Lurie of Dartmouth Hitchcock Medical Center. 
SPOT, a recently completed pragmatic trial that published its main outcomes earlier this year, compared 2 low-intensity outreach programs—care management and dialectical behavior therapy skills training—to usual care in adults at risk of self-harm or suicidal behavior. The study was administered by National Institute of Mental Health and was led by Dr. Greg Simon of the Kaiser Permanente Washington Health Research Institute. 