Iterative Token Evaluation and Refinement for Real-World
  Super-Resolution
Chaofeng Chen Shangchen Zhou Liang Liao Haoning Wu Wenxiu Sun Qiong Yan Weisi Lin
Real-world image super-resolution (RWSR) is a long-standing problem as
low-quality (LQ) images often have complex and unidentified degradations.
Existing methods such as Generative Adversarial Networks (GANs) or continuous
diffusion models present their own issues including GANs being difficult to
train while continuous diffusion models requiring numerous inference steps. In
this paper, we propose an Iterative Token Evaluation and Refinement (ITER)
framework for RWSR, which utilizes a discrete diffusion model operating in the
discrete token representation space, i.e., indexes of features extracted from a
VQGAN codebook pre-trained with high-quality (HQ) images. We show that ITER is
easier to train than GANs and more efficient than continuous diffusion models.
Specifically, we divide RWSR into two sub-tasks, i.e., distortion removal and
texture generation. Distortion removal involves simple HQ token prediction with
LQ images, while texture generation uses a discrete diffusion model to
iteratively refine the distortion removal output with a token refinement
network. In particular, we propose to include a token evaluation network in the
discrete diffusion process. It learns to evaluate which tokens are good
restorations and helps to improve the iterative refinement results. Moreover,
the evaluation network can first check status of the distortion removal output
and then adaptively select total refinement steps needed, thereby maintaining a
good balance between distortion removal and texture generation. Extensive
experimental results show that ITER is easy to train and performs well within
just 8 iterative steps. Our codes will be available publicly.