• Evaluation of AI chatbot respo...

*Result*: Evaluation of AI chatbot responses to brachytherapy frequently asked questions.

Title:
Evaluation of AI chatbot responses to brachytherapy frequently asked questions.
Authors:
Kouzy R; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Ibrahim I; Georgetown University School of Medicine, Washington, DC., Nemr P; Baylor College of Medicine, Houston, TX., Vinjamuri S; Gandhi Medical College, Secunderabad, Telangana, India., Ballout B; Texas College of Osteopathic Medicine, University of North Texas Health Science Center, Fort Worth, TX., Diaz JSGG; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Figueroa DN; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Alam MBE; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Kouzi ZE; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Hassanzadeh C; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Mohamad O; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Weil C; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Colbert L; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX., Klopp A; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX. Electronic address: aklopp@mdanderson.org.
Source:
Brachytherapy [Brachytherapy] 2026 Mar-Apr; Vol. 25 (2), pp. 275-282. Date of Electronic Publication: 2025 Dec 16.
Publication Type:
Journal Article
Language:
English
Journal Info:
Publisher: Elsevier, 2002- Country of Publication: United States NLM ID: 101137600 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1873-1449 (Electronic) Linking ISSN: 15384721 NLM ISO Abbreviation: Brachytherapy Subsets: MEDLINE
Imprint Name(s):
Original Publication: New York, NY : Elsevier, 2002-
MeSH Terms:
Brachytherapy*/methods , Artificial Intelligence*, Humans ; Comprehension ; Reproducibility of Results ; Generative Artificial Intelligence
Contributed Indexing:
Keywords: Artificial intelligence; Brachytherapy; Large Language Models; Patient education
Entry Date(s):
Date Created: 20251217 Date Completed: 20260223 Latest Revision: 20260223
Update Code:
20260224
DOI:
10.1016/j.brachy.2025.10.005
PMID:
41407567
Database:
MEDLINE

*Further Information*

*Purpose: Patients are increasingly using artificial intelligence (AI) chatbots for health information. Evaluating their reliability for specialized topics, such as brachytherapy, is crucial for guiding their safe use. We assessed a readily accessible AI chatbot's suitability for answering frequently asked questions (FAQ) related to brachytherapy.
Methods: We compared responses from an AI chatbot (ChatGPT 4o-mini) against gold standard (GS) authoritative sources for 10 brachytherapy frequently asked questions. Four blinded board-certified brachytherapy experts evaluated 80 response pairs using metrics, including accuracy, clinical appropriateness, readability, and tone. Five simulated patient personas with varying literacy levels were used to assess helpfulness, readability, and emotional tone. The objective readability metrics were also calculated.
Results: Experts rated the AI chatbot higher for accuracy (75% highly/mostly accurate vs. 50% for GS) and appropriateness (77% vs 55%), although inaccuracies were noted in both sources in a blinded review. Simulated patients preferred GS responses (62% vs. 34%), particularly lower-literacy personas, citing better perceived readability (92% easy/very easy vs. 44% for AI) and a more reassuring tone (42% vs. 24% for AI). Objective analysis confirmed that both sources significantly exceeded the recommended reading levels (e.g., >12th grade Flesch-Kincaid), with AI responses being substantially longer. Performance varied considerably across individual questions for both AI and GS sources.
Conclusions: In this blinded cross-sectional evaluation, a publicly available AI chatbot provided accurate responses to brachytherapy-related FAQs. However, further development and validation focused on accessibility, trustworthiness, and user-centered design are required before these tools can be safely and effectively integrated into patient-care workflows.
(Copyright © 2025. Published by Elsevier Inc.)*