Deep Networks as Denoising Algorithms: Sample-Efficient Learning of
  Diffusion Models in High-Dimensional Graphical Models
Song Mei Yuchen Wu
We investigate the approximation efficiency of score functions by deep neural
networks in diffusion-based generative modeling. While existing approximation
theories utilize the smoothness of score functions, they suffer from the curse
of dimensionality for intrinsically high-dimensional data. This limitation is
pronounced in graphical models such as Markov random fields, common for image
distributions, where the approximation efficiency of score functions remains
unestablished.
  To address this, we observe score functions can often be well-approximated in
graphical models through variational inference denoising algorithms.
Furthermore, these algorithms are amenable to efficient neural network
representation. We demonstrate this in examples of graphical models, including
Ising models, conditional Ising models, restricted Boltzmann machines, and
sparse encoding models. Combined with off-the-shelf discretization error bounds
for diffusion-based sampling, we provide an efficient sample complexity bound
for diffusion-based generative modeling when the score function is learned by
deep neural networks.