A Topic-aware Summarization Framework with Different Modal Side
  Information
Xiuying Chen Mingzhe Li Shen Gao Xin Cheng Qiang Yang Qishen Zhang Xin Gao Xiangliang Zhang
Automatic summarization plays an important role in the exponential document
growth on the Web. On content websites such as CNN.com and WikiHow.com, there
often exist various kinds of side information along with the main document for
attention attraction and easier understanding, such as videos, images, and
queries. Such information can be used for better summarization, as they often
explicitly or implicitly mention the essence of the article. However, most of
the existing side-aware summarization methods are designed to incorporate
either single-modal or multi-modal side information, and cannot effectively
adapt to each other. In this paper, we propose a general summarization
framework, which can flexibly incorporate various modalities of side
information. The main challenges in designing a flexible summarization model
with side information include: (1) the side information can be in textual or
visual format, and the model needs to align and unify it with the document into
the same semantic space, (2) the side inputs can contain information from
various aspects, and the model should recognize the aspects useful for
summarization. To address these two challenges, we first propose a unified
topic encoder, which jointly discovers latent topics from the document and
various kinds of side information. The learned topics flexibly bridge and guide
the information flow between multiple inputs in a graph encoder through a
topic-aware interaction. We secondly propose a triplet contrastive learning
mechanism to align the single-modal or multi-modal information into a unified
semantic space, where the summary quality is enhanced by better understanding
the document and side information. Results show that our model significantly
surpasses strong baselines on three public single-modal or multi-modal
benchmark summarization datasets.