Leveraging Large Language Models to Extract Information on Substance Use
  Disorder Severity from Clinical Notes: A Zero-shot Learning Approach
Maria Mahbub Gregory M. Dams Sudarshan Srinivasan Caitlin Rizy Ioana Danciu Jodie Trafton Kathryn Knight
Substance use disorder (SUD) poses a major concern due to its detrimental
effects on health and society. SUD identification and treatment depend on a
variety of factors such as severity, co-determinants (e.g., withdrawal
symptoms), and social determinants of health. Existing diagnostic coding
systems used by American insurance providers, like the International
Classification of Diseases (ICD-10), lack granularity for certain diagnoses,
but clinicians will add this granularity (as that found within the Diagnostic
and Statistical Manual of Mental Disorders classification or DSM-5) as
supplemental unstructured text in clinical notes. Traditional natural language
processing (NLP) methods face limitations in accurately parsing such diverse
clinical language. Large Language Models (LLMs) offer promise in overcoming
these challenges by adapting to diverse language patterns. This study
investigates the application of LLMs for extracting severity-related
information for various SUD diagnoses from clinical notes. We propose a
workflow employing zero-shot learning of LLMs with carefully crafted prompts
and post-processing techniques. Through experimentation with Flan-T5, an
open-source LLM, we demonstrate its superior recall compared to the rule-based
approach. Focusing on 11 categories of SUD diagnoses, we show the effectiveness
of LLMs in extracting severity information, contributing to improved risk
assessment and treatment planning for SUD patients.