Enriching Disentanglement: Definitions to Metrics
Yivan Zhang Masashi Sugiyama
Disentangled representation learning is a challenging task that involves
separating multiple factors of variation in complex data. Although various
metrics for learning and evaluating disentangled representations have been
proposed, it remains unclear what these metrics truly quantify and how to
compare them. In this work, we study the definitions of disentanglement given
by first-order equational predicates and introduce a systematic approach for
transforming an equational definition into a compatible quantitative metric
based on enriched category theory. Specifically, we show how to replace (i)
equality with metric or divergence, (ii) logical connectives with order
operations, (iii) universal quantifier with aggregation, and (iv) existential
quantifier with the best approximation. Using this approach, we derive metrics
for measuring the desired properties of a disentangled representation extractor
and demonstrate their effectiveness on synthetic data. Our proposed approach
provides practical guidance for researchers in selecting appropriate evaluation
metrics and designing effective learning algorithms for disentangled
representation learning.