Evaluating ChatGPT ability to answer urinary tract Infection-Related questions - 01/06/24
, Esra Zerdali c, Omer Sarilar b, Oguzhan Yildiz b, Faruk Ozgor bHighlights |
• | Innovative Study on ChatGPT in Urology: This study is the first to evaluate the accuracy and proficiency of ChatGPT's responses on urinary tract infections (UTIs), a crucial topic in urology. |
• | High Accuracy and Reliability of ChatGPT: ChatGPT demonstrated impressive accuracy rates (96.2% for FAQs, otherwise termed public-based inquiries) and 89.7% for guideline-based inquiries on UTIs), showing its potential as a reliable information source in urology. |
• | Effective Tool for Patient Education: The study suggests that ChatGPT, with its high reproducibility rates (over 90%) for both FAQs and scientific guidelines, can be an effective tool for educating patients in urology and infectious disease clinics. |
• | Comparison with Traditional Internet Resources: Unlike common internet resources that often provide unregulated information, ChatGPT's ability to access and analyze numerous sources ensures higher accuracy and reliability in its responses. |
• | Broad Implications in Medical Practice: The study underscores the potential benefits of integrating AI tools like ChatGPT in medical practice, particularly in improving public awareness and patient compliance in treatment processes. |
Abstract |
Introduction |
For the first time, the accuracy and proficiency of ChatGPT answers on urogenital tract infection (UTIs) were evaluated.
Methods |
The study aimed to create two lists of questions: frequently asked questions (FAQs, public-based inquiries) on relevant topics, and questions based on guideline information (guideline-based inquiries). ChatGPT responses to FAQs and scientific questions were scored by two urologists and an infectious disease specialist. Quality and reliability of all ChatGPT answers were checked using the Global Quality Score (GQS). The reproducibility of ChatGPT answers was analyzed by asking each question twice.
Results |
All in all, 96.2 % of FAQs (75/78 inquiries) related to UTIs were correctly and adequately answered by ChatGPT, and scored GQS 5. None of the ChatGPT answers were classified as GQS 2 and GQS 1. Moreover, FAQs about cystitis, urethritis, and epididymo-orchitis were answered by ChatGPT with 100 % accuracy (GQS 5). ChatGPT answers for EAU urological infections guidelines showed that 61 (89.7 %), 5 (7.4 %), and 2 (2.9 %) ChatGPT responses were scored GQS 5, GQS 4, and GQS 3, respectively. None of the ChatGPT responses for EAU urological infections guidelines were categorized as GQS 2 and GQS 1. Comparison of mean GQS values of ChatGPT answers for FAQs and EAU urological guideline questions showed that ChatGPT was similarly able to respond to both question groups (p = 0.168). The ChatGPT response reproducibility rate was highest for the FAQ subgroups of cystitis, urethritis, and epididymo-orchitis (100 % for each subgroup).
Conclusion |
The present study showed that ChatGPT gave accurate and satisfactory answers for both public-based inquiries, and EAU urological infection guideline-based questions. Reproducibility of ChatGPT answers exceeded 90% for both FAQs and scientific questions.
Le texte complet de cet article est disponible en PDF.Keywords : Artificial intelligence, ChatGPT, Guideline, Infection, Urinary tract infection
Plan
Vol 54 - N° 4
Article 104884- juin 2024 Retour au numéro
Article précédent
- Prevention and management of VZV infection during pregnancy and the perinatal period
- Caroline Charlier, Olivia Anselem, Marion Caseris, Marie Lachâtre, Asmaa Tazi, Marine Driessen, Didier Pinquier, Chemsa Le Cœur, Aurélie Saunier, Mathilde Bergamelli, Roxane Gibert Vanspranghels, Anaïs Chosidow, Charles Cazanave, Sophie Alain, Karine Faure, André Birgy, François Dubos, Philippe Lesprit, Julie Guinaud, Robert Cohen, Jean-Winoc Decousser, Emmanuel Grimprel, Cyril Huissoud, Julie Blanc, Gilles Kayem, Fanny Vuotto, Christelle Vauloup-Fellous
- Prevalence of antimicrobial use and healthcare-associated infections in the UAE: Results from the first nationwide point-prevalence survey
- Najiba Abdulrazzaq, Ayman Chkhis
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?