A Statistical Analysis of Wasserstein Autoencoders for Intrinsically
  Low-dimensional Data
Saptarshi Chakraborty Peter L. Bartlett
Variational Autoencoders (VAEs) have gained significant popularity among
researchers as a powerful tool for understanding unknown distributions based on
limited samples. This popularity stems partly from their impressive performance
and partly from their ability to provide meaningful feature representations in
the latent space. Wasserstein Autoencoders (WAEs), a variant of VAEs, aim to
not only improve model efficiency but also interpretability. However, there has
been limited focus on analyzing their statistical guarantees. The matter is
further complicated by the fact that the data distributions to which WAEs are
applied - such as natural images - are often presumed to possess an underlying
low-dimensional structure within a high-dimensional feature space, which
current theory does not adequately account for, rendering known bounds
inefficient. To bridge the gap between the theory and practice of WAEs, in this
paper, we show that WAEs can learn the data distributions when the network
architectures are properly chosen. We show that the convergence rates of the
expected excess risk in the number of samples for WAEs are independent of the
high feature dimension, instead relying only on the intrinsic dimension of the
data distribution.