Disentangling shared and private latent factors in multimodal
  Variational Autoencoders
Kaspar Märtens Christopher Yau
Generative models for multimodal data permit the identification of latent
factors that may be associated with important determinants of observed data
heterogeneity. Common or shared factors could be important for explaining
variation across modalities whereas other factors may be private and important
only for the explanation of a single modality. Multimodal Variational
Autoencoders, such as MVAE and MMVAE, are a natural choice for inferring those
underlying latent factors and separating shared variation from private. In this
work, we investigate their capability to reliably perform this disentanglement.
In particular, we highlight a challenging problem setting where
modality-specific variation dominates the shared signal. Taking a cross-modal
prediction perspective, we demonstrate limitations of existing models, and
propose a modification how to make them more robust to modality-specific
variation. Our findings are supported by experiments on synthetic as well as
various real-world multi-omics data sets.