A Review of Change of Variable Formulas for Generative Modeling
Ullrich Köthe
Change-of-variables (CoV) formulas allow to reduce complicated probability
densities to simpler ones by a learned transformation with tractable Jacobian
determinant. They are thus powerful tools for maximum-likelihood learning,
Bayesian inference, outlier detection, model selection, etc. CoV formulas have
been derived for a large variety of model types, but this information is
scattered over many separate works. We present a systematic treatment from the
unifying perspective of encoder/decoder architectures, which collects 28 CoV
formulas in a single place, reveals interesting relationships between seemingly
diverse methods, emphasizes important distinctions that are not always clear in
the literature, and identifies surprising gaps for future research.