Controllable Topic-Focused Abstractive Summarization
Seyed Ali Bahrainian Martin Jaggi Carsten Eickhoff
Controlled abstractive summarization focuses on producing condensed versions
of a source article to cover specific aspects by shifting the distribution of
generated text towards a desired style, e.g., a set of topics. Subsequently,
the resulting summaries may be tailored to user-defined requirements. This
paper presents a new Transformer-based architecture capable of producing
topic-focused summaries. The architecture modifies the cross-attention
mechanism of the Transformer to bring topic-focus control to the generation
process while not adding any further parameters to the model. We show that our
model sets a new state of the art on the NEWTS dataset in terms of
topic-focused abstractive summarization as well as a topic-prevalence score.
Moreover, we show via extensive experiments that our proposed topical
cross-attention mechanism can be plugged into various Transformer models, such
as BART and T5, improving their performance on the CNN/Dailymail and XSum
benchmark datasets for abstractive summarization. This is achieved via
fine-tuning, without requiring training from scratch. Finally, we show through
human evaluation that our model generates more faithful summaries outperforming
the state-of-the-art Frost model.