Non-asymptotic Convergence of Discrete-time Diffusion Models: New
  Approach and Improved Rate
Yuchen Liang Peizhong Ju Yingbin Liang Ness Shroff
The denoising diffusion model emerges recently as a powerful generative
technique that converts noise into data. Theoretical convergence guarantee has
been mainly studied for continuous-time diffusion models, and has been obtained
for discrete-time diffusion models only for distributions with bounded support
in the literature. In this paper, we establish the convergence guarantee for
substantially larger classes of distributions under discrete-time diffusion
models and further improve the convergence rate for distributions with bounded
support. In particular, we first establish the convergence rates for both
smooth and general (possibly non-smooth) distributions having finite second
moment. We then specialize our results to a number of interesting classes of
distributions with explicit parameter dependencies, including distributions
with Lipschitz scores, Gaussian mixture distributions, and distributions with
bounded support. We further propose a novel accelerated sampler and show that
it improves the convergence rates of the corresponding regular sampler by
orders of magnitude with respect to all system parameters. For distributions
with bounded support, our result improves the dimensional dependence of the
previous convergence rate by orders of magnitude. Our study features a novel
analysis technique that constructs tilting factor representation of the
convergence error and exploits Tweedie's formula for handling Taylor expansion
power terms.