Infusing Knowledge into Large Language Models with Contextual Prompts
Kinshuk Vasisht Balaji Ganesan Vikas Kumar Vasudha Bhatnagar
Knowledge infusion is a promising method for enhancing Large Language Models
for domain-specific NLP tasks rather than pre-training models over large data
from scratch. These augmented LLMs typically depend on additional pre-training
or knowledge prompts from an existing knowledge graph, which is impractical in
many applications. In contrast, knowledge infusion directly from relevant
documents is more generalisable and alleviates the need for structured
knowledge graphs while also being useful for entities that are usually not
found in any knowledge graph. With this motivation, we propose a simple yet
generalisable approach for knowledge infusion by generating prompts from the
context in the input text. Our experiments show the effectiveness of our
approach which we evaluate by probing the fine-tuned LLMs.