Large Language Model Capabilities in Perioperative Risk Prediction and
  Prognostication
Philip Chung Christine T Fong Andrew M Walters Nima Aghaeepour Meliha Yetisgen Vikas N O'Reilly-Shah
We investigate whether general-domain large language models such as GPT-4
Turbo can perform risk stratification and predict post-operative outcome
measures using a description of the procedure and a patient's clinical notes
derived from the electronic health record. We examine predictive performance on
8 different tasks: prediction of ASA Physical Status Classification, hospital
admission, ICU admission, unplanned admission, hospital mortality, PACU Phase 1
duration, hospital duration, and ICU duration. Few-shot and chain-of-thought
prompting improves predictive performance for several of the tasks. We achieve
F1 scores of 0.50 for ASA Physical Status Classification, 0.81 for ICU
admission, and 0.86 for hospital mortality. Performance on duration prediction
tasks were universally poor across all prompt strategies. Current generation
large language models can assist clinicians in perioperative risk
stratification on classification tasks and produce high-quality natural
language summaries and explanations.