LLMs-based Few-Shot Disease Predictions using EHR: A Novel Approach
  Combining Predictive Agent Reasoning and Critical Agent Instruction
Hejie Cui Zhuocheng Shen Jieyu Zhang Hui Shao Lianhui Qin Joyce C. Ho Carl Yang
Electronic health records (EHRs) contain valuable patient data for
health-related prediction tasks, such as disease prediction. Traditional
approaches rely on supervised learning methods that require large labeled
datasets, which can be expensive and challenging to obtain. In this study, we
investigate the feasibility of applying Large Language Models (LLMs) to convert
structured patient visit data (e.g., diagnoses, labs, prescriptions) into
natural language narratives. We evaluate the zero-shot and few-shot performance
of LLMs using various EHR-prediction-oriented prompting strategies.
Furthermore, we propose a novel approach that utilizes LLM agents with
different roles: a predictor agent that makes predictions and generates
reasoning processes and a critic agent that analyzes incorrect predictions and
provides guidance for improving the reasoning of the predictor agent. Our
results demonstrate that with the proposed approach, LLMs can achieve decent
few-shot performance compared to traditional supervised learning methods in
EHR-based disease predictions, suggesting its potential for health-oriented
applications.