Accelerating Diffusion Models for Inverse Problems through Shortcut
  Sampling
Gongye Liu Haoze Sun Jiayi Li Fei Yin Yujiu Yang
Recently, diffusion models have demonstrated a remarkable ability to solve
inverse problems in an unsupervised manner. Existing methods mainly focus on
modifying the posterior sampling process while neglecting the potential of the
forward process. In this work, we propose Shortcut Sampling for Diffusion
(SSD), a novel pipeline for solving inverse problems. Instead of initiating
from random noise, the key concept of SSD is to find the "Embryo", a
transitional state that bridges the measurement image y and the restored image
x. By utilizing the "shortcut" path of "input-Embryo-output", SSD can achieve
precise and fast restoration. To obtain the Embryo in the forward process, We
propose Distortion Adaptive Inversion (DA Inversion). Moreover, we apply back
projection and attention injection as additional consistency constraints during
the generation process. Experimentally, we demonstrate the effectiveness of SSD
on several representative tasks, including super-resolution, deblurring, and
colorization. Compared to state-of-the-art zero-shot methods, our method
achieves competitive results with only 30 NFEs. Moreover, SSD with 100 NFEs can
outperform state-of-the-art zero-shot methods in certain tasks.