Scaling and Masking: A New Paradigm of Data Sampling for Image and Video
  Quality Assessment
Yongxu Liu Yinghui Quan Guoyao Xiao Aobo Li Jinjian Wu
Quality assessment of images and videos emphasizes both local details and
global semantics, whereas general data sampling methods (e.g., resizing,
cropping or grid-based fragment) fail to catch them simultaneously. To address
the deficiency, current approaches have to adopt multi-branch models and take
as input the multi-resolution data, which burdens the model complexity. In this
work, instead of stacking up models, a more elegant data sampling method (named
as SAMA, scaling and masking) is explored, which compacts both the local and
global content in a regular input size. The basic idea is to scale the data
into a pyramid first, and reduce the pyramid into a regular data dimension with
a masking strategy. Benefiting from the spatial and temporal redundancy in
images and videos, the processed data maintains the multi-scale characteristics
with a regular input size, thus can be processed by a single-branch model. We
verify the sampling method in image and video quality assessment. Experiments
show that our sampling method can improve the performance of current
single-branch models significantly, and achieves competitive performance to the
multi-branch models without extra model complexity. The source code will be
available at https://github.com/Sissuire/SAMA.