Semi-Implicit Variational Inference via Score Matching
Longlin Yu Cheng Zhang
Semi-implicit variational inference (SIVI) greatly enriches the
expressiveness of variational families by considering implicit variational
distributions defined in a hierarchical manner. However, due to the intractable
densities of variational distributions, current SIVI approaches often use
surrogate evidence lower bounds (ELBOs) or employ expensive inner-loop MCMC
runs for unbiased ELBOs for training. In this paper, we propose SIVI-SM, a new
method for SIVI based on an alternative training objective via score matching.
Leveraging the hierarchical structure of semi-implicit variational families,
the score matching objective allows a minimax formulation where the intractable
variational densities can be naturally handled with denoising score matching.
We show that SIVI-SM closely matches the accuracy of MCMC and outperforms
ELBO-based SIVI methods in a variety of Bayesian inference tasks.