Harnessing the Power of Generative AI for Clinical Summaries: Perspectives From Emergency Physicians - 18/07/24
, Rebecca Wolf, BSc c, Ronen Rozenblum, MPH, PhD d, e, Jessica K. Creedon, MD c, d, Susan C. Lipsett, MD c, d, Todd W. Lyons, MD, MPH c, d, Kenneth A. Michelson, MD, MPH f, Kelsey A. Miller, MD, EdM c, d, Daniel J. Shapiro, MD g, Ben Y. Reis, PhD a, d, Andrew M. Fine, MD, MPH c, dAbstract |
Study objective |
The workload of clinical documentation contributes to health care costs and professional burnout. The advent of generative artificial intelligence language models presents a promising solution. The perspective of clinicians may contribute to effective and responsible implementation of such tools. This study sought to evaluate 3 uses for generative artificial intelligence for clinical documentation in pediatric emergency medicine, measuring time savings, effort reduction, and physician attitudes and identifying potential risks and barriers.
Methods |
This mixed-methods study was performed with 10 pediatric emergency medicine attending physicians from a single pediatric emergency department. Participants were asked to write a supervisory note for 4 clinical scenarios, with varying levels of complexity, twice without any assistance and twice with the assistance of ChatGPT Version 4.0. Participants evaluated 2 additional ChatGPT-generated clinical summaries: a structured handoff and a visit summary for a family written at an 8th grade reading level. Finally, a semistructured interview was performed to assess physicians’ perspective on the use of ChatGPT in pediatric emergency medicine. Main outcomes and measures included between subjects’ comparisons of the effort and time taken to complete the supervisory note with and without ChatGPT assistance. Effort was measured using a self-reported Likert scale of 0 to 10. Physicians’ scoring of and attitude toward the ChatGPT-generated summaries were measured using a 0 to 10 Likert scale and open-ended questions. Summaries were scored for completeness, accuracy, efficiency, readability, and overall satisfaction. A thematic analysis was performed to analyze the content of the open-ended questions and to identify key themes.
Results |
ChatGPT yielded a 40% reduction in time and a 33% decrease in effort for supervisory notes in intricate cases, with no discernible effect on simpler notes. ChatGPT-generated summaries for structured handoffs and family letters were highly rated, ranging from 7.0 to 9.0 out of 10, and most participants favored their inclusion in clinical practice. However, there were several critical reservations, out of which a set of general recommendations for applying ChatGPT to clinical summaries was formulated.
Conclusion |
Pediatric emergency medicine attendings in our study perceived that ChatGPT can deliver high-quality summaries while saving time and effort in many scenarios, but not all.
Le texte complet de cet article est disponible en PDF.Plan
| Supervising editor: Stephen Schenkel, MD, MPP. Specific detailed information about possible conflict of interest for individual editors is available at editors. |
|
| Author contributions: YBC, RR, BYR, and AMF conceptualized and designed the study, carried out the data analysis, drafted the initial manuscript, and reviewed and revised the final manuscript. JKC, SL, TL, KM, KM, and DJS reviewed study results and participated in the drafting and review of the final manuscript. YBC takes responsibility for the manuscript as a whole. |
|
| Data sharing statement: The data that support the findings of this study are available from the corresponding author upon reasonable request. |
|
| Authorship: All authors attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. |
|
| Funding and support: By Annals’ policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have declared that no competing interests exist. |
|
| Please see page 129 for the Editor’s Capsule Summary of this article. |
|
| A podcast for this article is available at www.annemergmed.com. |
|
| Readers: click on the link to go directly to a survey in which you can provide DYGWSNM to Annals on this particular article. |
Vol 84 - N° 2
P. 128-138 - août 2024 Retour au numéro
Article précédent
- Development and Validation of a Natural Language Processing Model to Identify Low-Risk Pulmonary Embolism in Real Time to Facilitate Safe Outpatient Management
- Krunal D. Amin, Elizabeth Hope Weissler, William Ratliff, Alexander E. Sullivan, Tara A. Holder, Cathleen Bury, Samuel Francis, Brent Jason Theiling, Bradley Hintze, Michael Gao, Marshall Nichols, Suresh Balu, William Schuyler Jones, Mark Sendak
- The AI Future of Emergency Medicine
- Robert J. Petrella
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?