Perception-Distortion Balanced Super-Resolution: A Multi-Objective
  Optimization Perspective
Lingchen Sun Jie Liang Shuaizheng Liu Hongwei Yong Lei Zhang
High perceptual quality and low distortion degree are two important goals in
image restoration tasks such as super-resolution (SR). Most of the existing SR
methods aim to achieve these goals by minimizing the corresponding yet
conflicting losses, such as the $\ell_1$ loss and the adversarial loss.
Unfortunately, the commonly used gradient-based optimizers, such as Adam, are
hard to balance these objectives due to the opposite gradient decent directions
of the contradictory losses. In this paper, we formulate the
perception-distortion trade-off in SR as a multi-objective optimization problem
and develop a new optimizer by integrating the gradient-free evolutionary
algorithm (EA) with gradient-based Adam, where EA and Adam focus on the
divergence and convergence of the optimization directions respectively. As a
result, a population of optimal models with different perception-distortion
preferences is obtained. We then design a fusion network to merge these models
into a single stronger one for an effective perception-distortion trade-off.
Experiments demonstrate that with the same backbone network, the
perception-distortion balanced SR model trained by our method can achieve
better perceptual quality than its competitors while attaining better
reconstruction fidelity. Codes and models can be found at
https://github.com/csslc/EA-Adam.