LLMs in Biomedicine: A study on clinical Named Entity Recognition
Masoud Monajatipoor Jiaxin Yang Joel Stremmel Melika Emami Fazlolah Mohaghegh Mozhdeh Rouhsedaghat Kai-Wei Chang
Large Language Models (LLMs) demonstrate remarkable versatility in various
NLP tasks but encounter distinct challenges in biomedicine due to medical
language complexities and data scarcity. This paper investigates the
application of LLMs in the medical domain by exploring strategies to enhance
their performance for the Named-Entity Recognition (NER) task. Specifically,
our study reveals the importance of meticulously designed prompts in
biomedicine. Strategic selection of in-context examples yields a notable
improvement, showcasing ~15-20\% increase in F1 score across all benchmark
datasets for few-shot clinical NER. Additionally, our findings suggest that
integrating external resources through prompting strategies can bridge the gap
between general-purpose LLM proficiency and the specialized demands of medical
NER. Leveraging a medical knowledge base, our proposed method inspired by
Retrieval-Augmented Generation (RAG) can boost the F1 score of LLMs for
zero-shot clinical NER. We will release the code upon publication.