Narrative Feature or Structured Feature? A Study of Large Language
  Models to Identify Cancer Patients at Risk of Heart Failure
Ziyi Chen Mengyuan Zhang Mustafa Mohammed Ahmed Yi Guo Thomas J. George Jiang Bian Yonghui Wu
Cancer treatments are known to introduce cardiotoxicity, negatively impacting
outcomes and survivorship. Identifying cancer patients at risk of heart failure
(HF) is critical to improving cancer treatment outcomes and safety. This study
examined machine learning (ML) models to identify cancer patients at risk of HF
using electronic health records (EHRs), including traditional ML, Time-Aware
long short-term memory (T-LSTM), and large language models (LLMs) using novel
narrative features derived from the structured medical codes. We identified a
cancer cohort of 12,806 patients from the University of Florida Health,
diagnosed with lung, breast, and colorectal cancers, among which 1,602
individuals developed HF after cancer. The LLM, GatorTron-3.9B, achieved the
best F1 scores, outperforming the traditional support vector machines by 39%,
the T-LSTM deep learning model by 7%, and a widely used transformer model,
BERT, by 5.6%. The analysis shows that the proposed narrative features
remarkably increased feature density and improved performance.