Boosting Summarization with Normalizing Flows and Aggressive Training
Yu Yang Xiaotong Shen
This paper presents FlowSUM, a normalizing flows-based variational
encoder-decoder framework for Transformer-based summarization. Our approach
tackles two primary challenges in variational summarization: insufficient
semantic information in latent representations and posterior collapse during
training. To address these challenges, we employ normalizing flows to enable
flexible latent posterior modeling, and we propose a controlled alternate
aggressive training (CAAT) strategy with an improved gate mechanism.
Experimental results show that FlowSUM significantly enhances the quality of
generated summaries and unleashes the potential for knowledge distillation with
minimal impact on inference time. Furthermore, we investigate the issue of
posterior collapse in normalizing flows and analyze how the summary quality is
affected by the training strategy, gate initialization, and the type and number
of normalizing flows used, offering valuable insights for future research.