Identifying Reasons for Contraceptive Switching from Real-World Data
  Using Large Language Models
Brenda Y. Miao Christopher YK Williams Ebenezer Chinedu-Eneh Travis Zack Emily Alsentzer Atul J. Butte Irene Y. Chen
Prescription contraceptives play a critical role in supporting women's
reproductive health. With nearly 50 million women in the United States using
contraceptives, understanding the factors that drive contraceptives selection
and switching is of significant interest. However, many factors related to
medication switching are often only captured in unstructured clinical notes and
can be difficult to extract. Here, we evaluate the zero-shot abilities of a
recently developed large language model, GPT-4 (via HIPAA-compliant Microsoft
Azure API), to identify reasons for switching between classes of contraceptives
from the UCSF Information Commons clinical notes dataset. We demonstrate that
GPT-4 can accurately extract reasons for contraceptive switching, outperforming
baseline BERT-based models with microF1 scores of 0.849 and 0.881 for
contraceptive start and stop extraction, respectively. Human evaluation of
GPT-4-extracted reasons for switching showed 91.4% accuracy, with minimal
hallucinations. Using extracted reasons, we identified patient preference,
adverse events, and insurance as key reasons for switching using unsupervised
topic modeling approaches. Notably, we also showed using our approach that
"weight gain/mood change" and "insurance coverage" are disproportionately found
as reasons for contraceptive switching in specific demographic populations. Our
code and supplemental data are available at
https://github.com/BMiao10/contraceptive-switching.