Assertion Detection Large Language Model In-context Learning LoRA
  Fine-tuning
Yuelyu Ji Zeshui Yu Yanshan Wang
In this study, we aim to address the task of assertion detection when
extracting medical concepts from clinical notes, a key process in clinical
natural language processing (NLP). Assertion detection in clinical NLP usually
involves identifying assertion types for medical concepts in the clinical text,
namely certainty (whether the medical concept is positive, negated, possible,
or hypothetical), temporality (whether the medical concept is for present or
the past history), and experiencer (whether the medical concept is described
for the patient or a family member). These assertion types are essential for
healthcare professionals to quickly and clearly understand the context of
medical conditions from unstructured clinical texts, directly influencing the
quality and outcomes of patient care. Although widely used, traditional
methods, particularly rule-based NLP systems and machine learning or deep
learning models, demand intensive manual efforts to create patterns and tend to
overlook less common assertion types, leading to an incomplete understanding of
the context. To address this challenge, our research introduces a novel
methodology that utilizes Large Language Models (LLMs) pre-trained on a vast
array of medical data for assertion detection. We enhanced the current method
with advanced reasoning techniques, including Tree of Thought (ToT), Chain of
Thought (CoT), and Self-Consistency (SC), and refine it further with Low-Rank
Adaptation (LoRA) fine-tuning. We first evaluated the model on the i2b2 2010
assertion dataset. Our method achieved a micro-averaged F-1 of 0.89, with 0.11
improvements over the previous works. To further assess the generalizability of
our approach, we extended our evaluation to a local dataset that focused on
sleep concept extraction. Our approach achieved an F-1 of 0.74, which is 0.31
higher than the previous method.