Biomedical knowledge graph-enhanced prompt generation for large language
  models
Karthik Soman Peter W Rose John H Morris Rabia E Akbas Brett Smith Braian Peetoom Catalina Villouta-Reyes Gabriel Cerono Yongmei Shi Angela Rizk-Jackson Sharat Israni Charlotte A Nelson Sui Huang Sergio E Baranzini
Large Language Models (LLMs) have been driving progress in AI at an
unprecedented rate, yet still face challenges in knowledge-intensive domains
like biomedicine. Solutions such as pre-training and domain-specific
fine-tuning add substantial computational overhead, and the latter require
domain-expertise. External knowledge infusion is task-specific and requires
model training. Here, we introduce a task-agnostic Knowledge Graph-based
Retrieval Augmented Generation (KG-RAG) framework by leveraging the massive
biomedical KG SPOKE with LLMs such as Llama-2-13b, GPT-3.5-Turbo and GPT-4, to
generate meaningful biomedical text rooted in established knowledge. KG-RAG
consistently enhanced the performance of LLMs across various prompt types,
including one-hop and two-hop prompts, drug repurposing queries, biomedical
true/false questions, and multiple-choice questions (MCQ). Notably, KG-RAG
provides a remarkable 71% boost in the performance of the Llama-2 model on the
challenging MCQ dataset, demonstrating the framework's capacity to empower
open-source models with fewer parameters for domain-specific questions.
Furthermore, KG-RAG enhanced the performance of proprietary GPT models, such as
GPT-3.5 which exhibited improvement over GPT-4 in context utilization on MCQ
data. Our approach was also able to address drug repurposing questions,
returning meaningful repurposing suggestions. In summary, the proposed
framework combines explicit and implicit knowledge of KG and LLM, respectively,
in an optimized fashion, thus enhancing the adaptability of general-purpose
LLMs to tackle domain-specific questions in a unified framework.