Event-Keyed Summarization
William Gantt Alexander Martin Pavlo Kuchmiichuk Aaron Steven White
We introduce event-keyed summarization (EKS), a novel task that marries
traditional summarization and document-level event extraction, with the goal of
generating a contextualized summary for a specific event, given a document and
an extracted event structure. We introduce a dataset for this task, MUCSUM,
consisting of summaries of all events in the classic MUC-4 dataset, along with
a set of baselines that comprises both pretrained LM standards in the
summarization literature, as well as larger frontier models. We show that
ablations that reduce EKS to traditional summarization or structure-to-text
yield inferior summaries of target events and that MUCSUM is a robust benchmark
for this task. Lastly, we conduct a human evaluation of both reference and
model summaries, and provide some detailed analysis of the results.