ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome
  Management
Angela Zhang Mert Yuksekgonul Joshua Guild James Zou Joseph C. Wu
Recent breakthroughs in large language models (LLMs) have led to their rapid
dissemination and widespread use. One early application has been to medicine,
where LLMs have been investigated to streamline clinical workflows and
facilitate clinical analysis and decision-making. However, a leading barrier to
the deployment of Artificial Intelligence (AI) and in particular LLMs has been
concern for embedded gender and racial biases. Here, we evaluate whether a
leading LLM, ChatGPT 3.5, exhibits gender and racial bias in clinical
management of acute coronary syndrome (ACS). We find that specifying patients
as female, African American, or Hispanic resulted in a decrease in guideline
recommended medical management, diagnosis, and symptom management of ACS. Most
notably, the largest disparities were seen in the recommendation of coronary
angiography or stress testing for the diagnosis and further intervention of ACS
and recommendation of high intensity statins. These disparities correlate with
biases that have been observed clinically and have been implicated in the
differential gender and racial morbidity and mortality outcomes of ACS and
coronary artery disease. Furthermore, we find that the largest disparities are
seen during unstable angina, where fewer explicit clinical guidelines exist.
Finally, we find that through asking ChatGPT 3.5 to explain its reasoning prior
to providing an answer, we are able to improve clinical accuracy and mitigate
instances of gender and racial biases. This is among the first studies to
demonstrate that the gender and racial biases that LLMs exhibit do in fact
affect clinical management. Additionally, we demonstrate that existing
strategies that improve LLM performance not only improve LLM performance in
clinical management, but can also be used to mitigate gender and racial biases.