Abstractive Text Summarization Using the BRIO Training Paradigm
Khang Nhut Lam Thieu Gia Doan Khang Thua Pham Jugal Kalita
Summary sentences produced by abstractive summarization models may be
coherent and comprehensive, but they lack control and rely heavily on reference
summaries. The BRIO training paradigm assumes a non-deterministic distribution
to reduce the model's dependence on reference summaries, and improve model
performance during inference. This paper presents a straightforward but
effective technique to improve abstractive summaries by fine-tuning pre-trained
language models, and training them with the BRIO paradigm. We build a text
summarization dataset for Vietnamese, called VieSum. We perform experiments
with abstractive summarization models trained with the BRIO paradigm on the
CNNDM and the VieSum datasets. The results show that the models, trained on
basic hardware, outperform all existing abstractive summarization models,
especially for Vietnamese.