Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot
  Question Answering
Sungho Ko Hyunjin Cho Hyungjoo Chae Jinyoung Yeo Dongha Lee
Recent studies have investigated utilizing Knowledge Graphs (KGs) to enhance
Quesetion Answering (QA) performance of Large Language Models (LLMs), yet
structured KG verbalization remains challengin. Existing methods, such as
triple-form or free-form textual conversion of triple-form facts, encounter
several issues. These include reduced evidence density due to duplicated
entities or relationships, and reduced evidence clarity due to an inability to
emphasize crucial evidence. To address these issues, we propose EFSum, an
Evidence-focused Fact Summarization framework for enhanced QA with
knowledge-augmented LLMs. We optimize an open-source LLM as a fact summarizer
through distillation and preference alignment. Our extensive experiments show
that EFSum improves LLM's zero-shot QA performance, and it is possible to
ensure both the helpfulness and faithfulness of the summary.