Towards a Unified Framework of Contrastive Learning for Disentangled
  Representations
Stefan Matthes Zhiwei Han Hao Shen
Contrastive learning has recently emerged as a promising approach for
learning data representations that discover and disentangle the explanatory
factors of the data. Previous analyses of such approaches have largely focused
on individual contrastive losses, such as noise-contrastive estimation (NCE)
and InfoNCE, and rely on specific assumptions about the data generating
process. This paper extends the theoretical guarantees for disentanglement to a
broader family of contrastive methods, while also relaxing the assumptions
about the data distribution. Specifically, we prove identifiability of the true
latents for four contrastive losses studied in this paper, without imposing
common independence assumptions. The theoretical findings are validated on
several benchmark datasets. Finally, practical limitations of these methods are
also investigated.