Learning variational autoencoders via MCMC speed measures
Marcel Hirt Vasileios Kreouzis Petros Dellaportas
Variational autoencoders (VAEs) are popular likelihood-based generative
models which can be efficiently trained by maximizing an Evidence Lower Bound
(ELBO). There has been much progress in improving the expressiveness of the
variational distribution to obtain tighter variational bounds and increased
generative performance. Whilst previous work has leveraged Markov chain Monte
Carlo (MCMC) methods for the construction of variational densities,
gradient-based methods for adapting the proposal distributions for deep latent
variable models have received less attention. This work suggests an
entropy-based adaptation for a short-run Metropolis-adjusted Langevin (MALA) or
Hamiltonian Monte Carlo (HMC) chain while optimising a tighter variational
bound to the log-evidence. Experiments show that this approach yields higher
held-out log-likelihoods as well as improved generative metrics. Our implicit
variational density can adapt to complicated posterior geometries of latent
hierarchical representations arising in hierarchical VAEs.