We investigate the properties of multidimensional probability distributions in the context of latent space prior distributions of implicit generative models. Our work revolves around the phenomena arising while decoding linear interpolations between two random latent vectors -- regions of latent space in close proximity to the origin of the space are oversampled, which restricts the usability of linear interpolations as a tool to analyse the latent space. We show that the distribution mismatch can be eliminated completely by a proper choice of the latent probability distribution or using non-linear interpolations. We prove that there is a trade off between the interpolation being linear, and the latent distribution having even the most basic properties required for stable training, such as finite mean. We use the multidimensional Cauchy distribution as an example of the prior distribution, and also provide a general method of creating non-linear interpolations, that is easily applicable to a large family of commonly used latent distributions. Generative latent variable models have grown to be a very popular research topic, with Variational Auto-Encoders (VAEs) BID8 and Generative Adversarial Networks (GANs) BID4 gaining a lot of interest in the last few years. VAEs use a stochastic encoder network to embed input data in a typically lower dimensional space, using a conditional probability distribution p(z|x) over possible latent space codes z ∈ R D . A stochastic decoder network is then used to reconstruct the original sample. GANs, on the other hand, use a generator network that creates data samples from noise z ∼ p(z), where p(z) is a fixed prior distribution, and train a discriminator network jointly to distinguish between real and generated data. Both of these model families require a probability distribution to be defined on the latent space. The most popular variants are the multidimensional normal distribution and the uniform distribution on the zero-centred hypercube. Given a trained model, studying the structure of the latent space is a common way to measure generator capabilities.
We theoretically prove that linear interpolations are unsuitable for analysis of trained implicit generative models.  Studies the problem of when the linear interpolant between two random variables follows the same distribution, related to prior distribution of an implicit generative model This work asks how to interpolate in the latent space given a latent variable model.