Do You Hear The People Sing? Key Point Analysis via Iterative Clustering
  and Abstractive Summarisation
Hao Li Viktor Schlegel Riza Batista-Navarro Goran Nenadic
Argument summarisation is a promising but currently under-explored field.
Recent work has aimed to provide textual summaries in the form of concise and
salient short texts, i.e., key points (KPs), in a task known as Key Point
Analysis (KPA). One of the main challenges in KPA is finding high-quality key
point candidates from dozens of arguments even in a small corpus. Furthermore,
evaluating key points is crucial in ensuring that the automatically generated
summaries are useful. Although automatic methods for evaluating summarisation
have considerably advanced over the years, they mainly focus on sentence-level
comparison, making it difficult to measure the quality of a summary (a set of
KPs) as a whole. Aggravating this problem is the fact that human evaluation is
costly and unreproducible. To address the above issues, we propose a two-step
abstractive summarisation framework based on neural topic modelling with an
iterative clustering procedure, to generate key points which are aligned with
how humans identify key points. Our experiments show that our framework
advances the state of the art in KPA, with performance improvement of up to 14
(absolute) percentage points, in terms of both ROUGE and our own proposed
evaluation metrics. Furthermore, we evaluate the generated summaries using a
novel set-based evaluation toolkit. Our quantitative analysis demonstrates the
effectiveness of our proposed evaluation metrics in assessing the quality of
generated KPs. Human evaluation further demonstrates the advantages of our
approach and validates that our proposed evaluation metric is more consistent
with human judgment than ROUGE scores.