A Causal Ordering Prior for Unsupervised Representation Learning
Avinash Kori Pedro Sanchez Konstantinos Vilouras Ben Glocker Sotirios A. Tsaftaris
Unsupervised representation learning with variational inference relies
heavily on independence assumptions over latent variables. Causal
representation learning (CRL), however, argues that factors of variation in a
dataset are, in fact, causally related. Allowing latent variables to be
correlated, as a consequence of causal relationships, is more realistic and
generalisable. So far, provably identifiable methods rely on: auxiliary
information, weak labels, and interventional or even counterfactual data.
Inspired by causal discovery with functional causal models, we propose a fully
unsupervised representation learning method that considers a data generation
process with a latent additive noise model (ANM). We encourage the latent space
to follow a causal ordering via loss function based on the Hessian of the
latent distribution.