BlindDiff: Empowering Degradation Modelling in Diffusion Models for
  Blind Image Super-Resolution
Feng Li Yixuan Wu Zichao Liang Runmin Cong Huihui Bai Yao Zhao Meng Wang
Diffusion models (DM) have achieved remarkable promise in image
super-resolution (SR). However, most of them are tailored to solving non-blind
inverse problems with fixed known degradation settings, limiting their
adaptability to real-world applications that involve complex unknown
degradations. In this work, we propose BlindDiff, a DM-based blind SR method to
tackle the blind degradation settings in SISR. BlindDiff seamlessly integrates
the MAP-based optimization into DMs, which constructs a joint distribution of
the low-resolution (LR) observation, high-resolution (HR) data, and degradation
kernels for the data and kernel priors, and solves the blind SR problem by
unfolding MAP approach along with the reverse process. Unlike most DMs,
BlindDiff firstly presents a modulated conditional transformer (MCFormer) that
is pre-trained with noise and kernel constraints, further serving as a
posterior sampler to provide both priors simultaneously. Then, we plug a simple
yet effective kernel-aware gradient term between adjacent sampling iterations
that guides the diffusion model to learn degradation consistency knowledge.
This also enables to joint refine the degradation model as well as HR images by
observing the previous denoised sample. With the MAP-based reverse diffusion
process, we show that BlindDiff advocates alternate optimization for blur
kernel estimation and HR image restoration in a mutual reinforcing manner.
Experiments on both synthetic and real-world datasets show that BlindDiff
achieves the state-of-the-art performance with significant model complexity
reduction compared to recent DM-based methods. Code will be available at
\url{https://github.com/lifengcs/BlindDiff}