Assessing distance betweeen the true and the sample distribution is a key component of many state of the art generative models, such as Wasserstein Autoencoder (WAE). Inspired by prior work on Sliced-Wasserstein Autoencoders (SWAE) and
 kernel smoothing we construct a new generative model – Cramer-Wold AutoEncoder (CWAE). CWAE cost function, based on introduced Cramer-Wold distance between samples, has a simple closed-form in the case of normal prior. As a consequence, while simplifying the optimization procedure (no need of sampling necessary to evaluate the distance function in the training loop), CWAE performance matches quantitatively and qualitatively that of WAE-MMD (WAE using maximum mean discrepancy based distance function) and often improves upon SWAE. One of the crucial aspects in construction of generative models is devising effective method for computing and minimizing distance between the true and the model distribution. Originally in Variational Autencoder (VAE) BID10 this computation was carried out using variational methods. An important improvement was brought by the introduction of Wasserstein metric BID14 and the construction of WAE-GAN and WAE-MMD models, which relax the need for variational methods. WAE-GAN requires a separate optimization problem to be solved to approximate the used divergence measure, while in WAE-MMD the discriminator has the closed-form obtained from a characteristic kernel, i.e. one that is injective on distributions BID12 . A recent contribution to this trend of simplifying the construction of generative models is Sliced-Wasserstein Autoencoder (SWAE, BID11 ), where a significantly simpler AutoEncoder based model based on Wasserstein distance is proposed. The main innovation of SWAE was the introduction of the sliced-Wasserstein distance -a fast to estimate metric for comparing two distributions, based on the mean Wasserstein distance of one-dimensional projections. However, even in SWAE there is no close analytic formula that would enable computing the distance of the sample from the standard normal distribution. Consequently in SWAE two types of sampling are needed: (i) sampling from the prior distribution and (ii) sampling over one-dimensional projections.Our main contribution is introduction of the CramerWold distance between distributions, which has a closed-form for the distance of a sample from standard multivariate normal distribution. Its important feature is that it is given by a characteristic kernel which has a closed-form given by equation 7 for the product of radial Gaussians 1 . We use it to construct an AutoEncoder based generative model, called Cramer-Wold AutoEncoder (CWAE), in which the cost function, for a normal prior distribution, has a closed analytic formula. Thus In the paper we have presented a new autoencoder based generative model CWAE, which matches results of WAE-MMD, while using a cost function given by a simple closed analytic formula. We hope this result will encourage future work in developing simpler to optimize analogs of strong neural models.Crucial in the construction of CWAE is the use of the developed Cramer-Wold metric between samples and distributions, which can be effectively computed for Gaussian mixtures. As a consequence we obtain a reliable measure of the divergence from normality. Future work could explore use of the Cramer-Wold distance in other settings, in particular in adversarial models.
Inspired by prior work on Sliced-Wasserstein Autoencoders (SWAE) and kernel smoothing we construct a new generative model – Cramer-Wold AutoEncoder (CWAE). This paper proposes a WAE variant based on a new statistical distance between the encoded data distribution and the latent prior distribution Introduces a variation on the Wasserstein AudoEncoders which is a novel regularized auto-encoder architecture that proposes a specific choice of the divergence penalty This paper proposes the Cramer-Wold autoencoder, which uses the Cramer-Wold distance between two distributions based on the Cramer-Wold Theorem.