Do GANS (Generative Adversarial Nets) actually learn the target distribution? The foundational paper of Goodfellow et al. (2014) suggested they do, if they were given sufficiently large deep nets, sample size, and computation time. A recent theoretical analysis in Arora et al. (2017) raised doubts whether the same holds when discriminator has bounded size. It showed that the training objective can approach its optimum value even if the generated distribution has very low support. In other words, the training objective is unable to prevent mode collapse. The current paper makes two contributions. (1) It proposes a novel test for estimating support size using the birthday paradox of discrete probability. Using this  evidence is presented that well-known GANs approaches do learn distributions of fairly low support.   (2) It theoretically studies encoder-decoder GANs architectures (e.g., BiGAN/ALI), which were proposed to learn more meaningful features via GANs, and consequently to also solve the mode-collapse issue. Our result shows that such encoder-decoder training objectives also cannot guarantee learning of the full distribution because they cannot prevent serious mode collapse. More seriously, they cannot prevent learning meaningless codes for data, contrary to usual intuition. From the earliest papers on Generative Adversarial Networks the question has been raised whether or not they actually come close to learning the distribution they are trained with (henceforth refered to as the target distribution)? These methods train a generator deep net that converts a random seed into a realistic-looking image. Concurrently they train a discriminator deep net to discriminate between its output and real images, which in turn is used to produce gradient feedback to improve the generator net. In practice the generator deep net starts producing realistic outputs by the end, and the objective approaches its optimal value. But does this mean the deep net has learnt the target distribution of real images? Standard analysis introduced in BID3 shows that given "sufficiently large" generator and discriminator, sample size, and computation time the training does succeed in learning the underlying distribution arbitrarily closely (measured in JensenShannon divergence). But this does not settle the question of what happens with realistic sample and net sizes.Note that GANs differ from many previous methods for learning distributions in that they do not provide an estimate of a measure of distributional fit -e.g., perplexity score. Therefore researchers have probed their performance using surrogate qualitative tests, which were usually designed to rule out the most obvious failure mode of the training, namely, that the GAN has simply memorized the training data. One test checks the similarity of each generated image to the nearest images in the training set. Another takes two random seeds s 1 , s 2 that produced realistic images and checks the images produced using seeds lying on the line joining s 1 , s 2 . If such "interpolating" images are reasonable and original as well, then this may be taken as evidence that the generated distribution has many novel images. Yet other tests check for existence of semantically meaningful directions in the latent space, meaning that varying the seed along these directions leads to predictable changes e.g., (in case of images of human faces) changes in facial hair, or pose. A recent test proposed by BID12 checks the log-likelihoods of GANs using Annealed Importance Sampling, whose results indicate the mismatch between generator's distribution and the target distribution. BID8 proposed a method to trade-off between sample quality and sample diversity but they don't provide a clear definition or a quantitative metric of sample diversity.Recently a new theoretical analysis of GANs with finite sample sizes and finite discriminator size revealed the possibility that training objective can sometimes approach optimality even if the generator is far from having actually learnt the distribution. Specifically, if the discriminator has size p, then the training objective could be close to optimal even though the output distribution is supported on only O(p log p/ 2 ) images. By contrast one imagines that the target distribution usually must have very large support. For example, the set of all possible images of human faces (a frequent setting in GANs work) must involve all combinations of hair color/style, facial features, complexion, expression, pose, lighting, race, etc., and thus the possible set of images of faces approaches infinity. The above paper raises the possibility that the discriminator may be unable to meaningfully distinguish such a diverse target distribution from a trained distribution with fairly small support. Furthermore, the paper notes that this failure mode is different from the one usually feared, namely the generator memorizing training samples. The analysis of raises the possibility that the trained distribution has small support, and yet all its samples could be completely disjoint from the training samples.However, the above analysis was only a theoretical one, exhibiting a particular near-equilibrium solution that can happen from certain hyper-parameter combinations. It left open the possibility that real-life GANs training avoids such solutions thanks to some not-as-yet-understood property of SGD, or hyper-parameter choices. Thus further experimental investigation seems necessary. And yet it seems difficult at first sight to do such an empirical evaluation of the support size of a distribution: it is not humanly possible to go through hundreds of thousands of images, whereas automated tests of image similarity can be thrown off by small changes in lighting, pose etc.The current paper makes two important contributions. On the empirical side it introduces a new test for the support size of the trained distribution, and uses it to find that unfortunately these mode collapse problems do arise in many well-regarded GAN training methods. On the theoretical side we prove the limitations of encoder-decoder frameworks like BiGAN BID1 and Adversarially Learned Inference or ALI BID2 , which, inspired by autoencoder models, require the setup to learn an inference mechanism as well as a generative mechanism. The result of applies only to standard GAN training objectives (including JS and Wasserstein), but not to encoder-decoder setups. The clear hope in defining encoder-decoder setups is that the encoding mechanism "inverts" the generator and thus forces the generator to learn meaningful featurizations of data that are useful in downstream applications. In fact it has often been proposed that this need to learn meaningful featurizations will also solve the mode collapse problem: BID2 provide experiments on 2-dimensional mixtures of Gaussians suggesting this phenomenon. Our analysis shows not only that encoder-decoder training objectives cannot avoid mode collapse, but that they also cannot enforce learning of meaningful codes/features. The paper reveals gaps in current thinking about GANs, and hopes to stimulate further theoretical and empirical study. GANs research has always struggled with the issue of mode collapse, and recent theoretical analysis of shows that the GANs training objective is not capable of preventing mode collapse. This exhibits the existence of bad solutions in the optimization landscape. This in itself is not definitive, since existence of bad solutions is also known for the more traditional classification tasks , where heldout sets can nevertheless prove that a good solution has been reached. The difference in case of GANs is lack of an obvious way to establish that training succeeded.Our new Birthday Paradox test gives a new benchmark for testing the support size (i.e., diversity of images) in a distribution. Though it may appear weak, experiments using this test suggest that current GANs approaches, specifically, the ones that produce images of higher visual quality, do suffer mode collapse. Our rough experiments also suggest -again in line with the previous theoretical analysis-that the size of the distribution's support scales near-linearly with discriminator capacity.Researchers have raised the possibility that the best use of GANs is not distribution learning but feature learning. Encoder-decoder GAN architectures seem promising since they try to force the generator to use "meaningful" encodings of the image. While such architectures do exhibit slightly better diversity in our experiments, our theoretical result suggest that the the encoder-decoder objective is also unable to avoid mode collapse, furthermore, also fails to guarantee meaningful codes.
We propose a support size estimator of GANs's learned distribution to show they indeed suffer from mode collapse, and we prove that encoder-decoder GANs do not avoid the issue as well. The paper attempts to estimate the size of the support for solutions produced by typical GANs experimentally.  This paper proposes a clever new test based on the birthday paradox for measuring diversity in generated sample, with experiment results interpreted to mean that mode collapse is strong in a number of state-of-the-art generative models. The paper uses birthday paradox to show that some GAN architectures generate distributions with fairly low support.