Comparative Study of Domain Driven Terms Extraction Using Large Language
  Models
Sandeep Chataut Tuyen Do Bichar Dip Shrestha Gurung Shiva Aryal Anup Khanal Carol Lushbough Etienne Gnimpieba
Keywords play a crucial role in bridging the gap between human understanding
and machine processing of textual data. They are essential to data enrichment
because they form the basis for detailed annotations that provide a more
insightful and in-depth view of the underlying data. Keyword/domain driven term
extraction is a pivotal task in natural language processing, facilitating
information retrieval, document summarization, and content categorization. This
review focuses on keyword extraction methods, emphasizing the use of three
major Large Language Models(LLMs): Llama2-7B, GPT-3.5, and Falcon-7B. We
employed a custom Python package to interface with these LLMs, simplifying
keyword extraction. Our study, utilizing the Inspec and PubMed datasets,
evaluates the performance of these models. The Jaccard similarity index was
used for assessment, yielding scores of 0.64 (Inspec) and 0.21 (PubMed) for
GPT-3.5, 0.40 and 0.17 for Llama2-7B, and 0.23 and 0.12 for Falcon-7B. This
paper underlines the role of prompt engineering in LLMs for better keyword
extraction and discusses the impact of hallucination in LLMs on result
evaluation. It also sheds light on the challenges in using LLMs for keyword
extraction, including model complexity, resource demands, and optimization
techniques.