Multi-Query Focused Disaster Summarization via Instruction-Based
  Prompting
Philipp Seeberger Korbinian Riedhammer
Automatic summarization of mass-emergency events plays a critical role in
disaster management. The second edition of CrisisFACTS aims to advance disaster
summarization based on multi-stream fact-finding with a focus on web sources
such as Twitter, Reddit, Facebook, and Webnews. Here, participants are asked to
develop systems that can extract key facts from several disaster-related
events, which ultimately serve as a summary. This paper describes our method to
tackle this challenging task. We follow previous work and propose to use a
combination of retrieval, reranking, and an embarrassingly simple
instruction-following summarization. The two-stage retrieval pipeline relies on
BM25 and MonoT5, while the summarizer module is based on the open-source Large
Language Model (LLM) LLaMA-13b. For summarization, we explore a Question
Answering (QA)-motivated prompting approach and find the evidence useful for
extracting query-relevant facts. The automatic metrics and human evaluation
show strong results but also highlight the gap between open-source and
proprietary systems.