Hierarchical Semi-Implicit Variational Inference with Application to
  Diffusion Model Acceleration
Longlin Yu Tianyu Xie Yu Zhu Tong Yang Xiangyu Zhang Cheng Zhang
Semi-implicit variational inference (SIVI) has been introduced to expand the
analytical variational families by defining expressive semi-implicit
distributions in a hierarchical manner. However, the single-layer architecture
commonly used in current SIVI methods can be insufficient when the target
posterior has complicated structures. In this paper, we propose hierarchical
semi-implicit variational inference, called HSIVI, which generalizes SIVI to
allow more expressive multi-layer construction of semi-implicit distributions.
By introducing auxiliary distributions that interpolate between a simple base
distribution and the target distribution, the conditional layers can be trained
by progressively matching these auxiliary distributions one layer after
another. Moreover, given pre-trained score networks, HSIVI can be used to
accelerate the sampling process of diffusion models with the score matching
objective. We show that HSIVI significantly enhances the expressiveness of SIVI
on several Bayesian inference problems with complicated target distributions.
When used for diffusion model acceleration, we show that HSIVI can produce high
quality samples comparable to or better than the existing fast diffusion model
based samplers with a small number of function evaluations on various datasets.