On the Asymptotic Mean Square Error Optimality of Diffusion
  Probabilistic Models
Benedikt Fesl Benedikt Böck Florian Strasser Michael Baur Michael Joham Wolfgang Utschick
Diffusion probabilistic models (DPMs) have recently shown great potential for
denoising tasks. Despite their practical utility, there is a notable gap in
their theoretical understanding. This paper contributes novel theoretical
insights by rigorously proving the asymptotic convergence of a specific DPM
denoising strategy to the mean square error (MSE)-optimal conditional mean
estimator (CME) over a large number of diffusion steps. The studied DPM-based
denoiser shares the training procedure of DPMs but distinguishes itself by
forwarding only the conditional mean during the reverse inference process after
training. We highlight the unique perspective that DPMs are composed of an
asymptotically optimal denoiser while simultaneously inheriting a powerful
generator by switching re-sampling in the reverse process on and off. The
theoretical findings are validated by numerical results.