PartDiff: Image Super-resolution with Partial Diffusion Models
Kai Zhao Alex Ling Yu Hung Kaifeng Pang Haoxin Zheng Kyunghyun Sung
Denoising diffusion probabilistic models (DDPMs) have achieved impressive
performance on various image generation tasks, including image
super-resolution. By learning to reverse the process of gradually diffusing the
data distribution into Gaussian noise, DDPMs generate new data by iteratively
denoising from random noise. Despite their impressive performance,
diffusion-based generative models suffer from high computational costs due to
the large number of denoising steps.In this paper, we first observed that the
intermediate latent states gradually converge and become indistinguishable when
diffusing a pair of low- and high-resolution images. This observation inspired
us to propose the Partial Diffusion Model (PartDiff), which diffuses the image
to an intermediate latent state instead of pure random noise, where the
intermediate latent state is approximated by the latent of diffusing the
low-resolution image. During generation, Partial Diffusion Models start
denoising from the intermediate distribution and perform only a part of the
denoising steps. Additionally, to mitigate the error caused by the
approximation, we introduce "latent alignment", which aligns the latent between
low- and high-resolution images during training. Experiments on both magnetic
resonance imaging (MRI) and natural images show that, compared to plain
diffusion-based super-resolution methods, Partial Diffusion Models
significantly reduce the number of denoising steps without sacrificing the
quality of generation.