Learning with Noisy Low-Cost MOS for Image Quality Assessment via
  Dual-Bias Calibration
Lei Wang Qingbo Wu Desen Yuan King Ngi Ngan Hongliang Li Fanman Meng Linfeng Xu
Learning based image quality assessment (IQA) models have obtained impressive
performance with the help of reliable subjective quality labels, where mean
opinion score (MOS) is the most popular choice. However, in view of the
subjective bias of individual annotators, the labor-abundant MOS (LA-MOS)
typically requires a large collection of opinion scores from multiple
annotators for each image, which significantly increases the learning cost. In
this paper, we aim to learn robust IQA models from low-cost MOS (LC-MOS), which
only requires very few opinion scores or even a single opinion score for each
image. More specifically, we consider the LC-MOS as the noisy observation of
LA-MOS and enforce the IQA model learned from LC-MOS to approach the unbiased
estimation of LA-MOS. In this way, we represent the subjective bias between
LC-MOS and LA-MOS, and the model bias between IQA predictions learned from
LC-MOS and LA-MOS (i.e., dual-bias) as two latent variables with unknown
parameters. By means of the expectation-maximization based alternating
optimization, we can jointly estimate the parameters of the dual-bias, which
suppresses the misleading of LC-MOS via a gated dual-bias calibration (GDBC)
module. To the best of our knowledge, this is the first exploration of robust
IQA model learning from noisy low-cost labels. Theoretical analysis and
extensive experiments on four popular IQA datasets show that the proposed
method is robust toward different bias rates and annotation numbers and
significantly outperforms the other learning based IQA models when only LC-MOS
is available. Furthermore, we also achieve comparable performance with respect
to the other models learned with LA-MOS.