Posterior Consistency for Missing Data in Variational Autoencoders
Timur Sudak Sebastian Tschiatschek
We consider the problem of learning Variational Autoencoders (VAEs), i.e., a
type of deep generative model, from data with missing values. Such data is
omnipresent in real-world applications of machine learning because complete
data is often impossible or too costly to obtain. We particularly focus on
improving a VAE's amortized posterior inference, i.e., the encoder, which in
the case of missing data can be susceptible to learning inconsistent posterior
distributions regarding the missingness. To this end, we provide a formal
definition of posterior consistency and propose an approach for regularizing an
encoder's posterior distribution which promotes this consistency. We observe
that the proposed regularization suggests a different training objective than
that typically considered in the literature when facing missing values.
Furthermore, we empirically demonstrate that our regularization leads to
improved performance in missing value settings in terms of reconstruction
quality and downstream tasks utilizing uncertainty in the latent space. This
improved performance can be observed for many classes of VAEs including VAEs
equipped with normalizing flows.