Assessing the Usability of GutGPT: A Simulation Study of an AI Clinical
  Decision Support System for Gastrointestinal Bleeding Risk
Colleen Chan Kisung You Sunny Chung Mauro Giuffrè Theo Saarinen Niroop Rajashekar Yuan Pu Yeo Eun Shin Loren Laine Ambrose Wong René Kizilcec Jasjeet Sekhon Dennis Shung
Applications of large language models (LLMs) like ChatGPT have potential to
enhance clinical decision support through conversational interfaces. However,
challenges of human-algorithmic interaction and clinician trust are poorly
understood. GutGPT, a LLM for gastrointestinal (GI) bleeding risk prediction
and management guidance, was deployed in clinical simulation scenarios
alongside the electronic health record (EHR) with emergency medicine
physicians, internal medicine physicians, and medical students to evaluate its
effect on physician acceptance and trust in AI clinical decision support
systems (AI-CDSS). GutGPT provides risk predictions from a validated machine
learning model and evidence-based answers by querying extracted clinical
guidelines. Participants were randomized to GutGPT and an interactive
dashboard, or the interactive dashboard and a search engine. Surveys and
educational assessments taken before and after measured technology acceptance
and content mastery. Preliminary results showed mixed effects on acceptance
after using GutGPT compared to the dashboard or search engine but appeared to
improve content mastery based on simulation performance. Overall, this study
demonstrates LLMs like GutGPT could enhance effective AI-CDSS if implemented
optimally and paired with interactive interfaces.