Bias in Opinion Summarisation from Pre-training to Adaptation: A Case
  Study in Political Bias
Nannan Huang Haytham Fayek Xiuzhen Zhang
Opinion summarisation aims to summarise the salient information and opinions
presented in documents such as product reviews, discussion forums, and social
media texts into short summaries that enable users to effectively understand
the opinions therein. Generating biased summaries has the risk of potentially
swaying public opinion. Previous studies focused on studying bias in opinion
summarisation using extractive models, but limited research has paid attention
to abstractive summarisation models. In this study, using political bias as a
case study, we first establish a methodology to quantify bias in abstractive
models, then trace it from the pre-trained models to the task of summarising
social media opinions using different models and adaptation methods. We find
that most models exhibit intrinsic bias. Using a social media text
summarisation dataset and contrasting various adaptation methods, we find that
tuning a smaller number of parameters is less biased compared to standard
fine-tuning; however, the diversity of topics in training data used for
fine-tuning is critical.