On the Universality of Coupling-based Normalizing Flows
Felix Draxler Stefan Wahl Christoph Schnörr Ullrich Köthe
We present a novel theoretical framework for understanding the expressive
power of coupling-based normalizing flows such as RealNVP. Despite their
prevalence in scientific applications, a comprehensive understanding of
coupling flows remains elusive due to their restricted architectures. Existing
theorems fall short as they require the use of arbitrarily ill-conditioned
neural networks, limiting practical applicability. Additionally, we demonstrate
that these constructions inherently lead to volume-preserving flows, a property
which we show to be a fundamental constraint for expressivity. We propose a new
distributional universality theorem for coupling-based normalizing flows, which
overcomes several limitations of prior work. Our results support the general
wisdom that the coupling architecture is expressive and provide a nuanced view
for choosing the expressivity of coupling functions, bridging a gap between
empirical results and theoretical understanding.