In this paper, we propose a mix-generator generative adversarial networks (PGAN) model that works in parallel by mixing multiple disjoint generators to approximate a complex real distribution. In our model, we propose an adjustment component that collects all the generated data points from the generators, learns the boundary between each pair of generators, and provides error to separate the support of each of the generated distributions. To overcome the instability in a multiplayer game, a shrinkage adjustment component method is introduced to gradually reduce the boundary between generators during the training procedure. To address the linearly growing training time problem in a multiple generators model, we propose a method to train the generators in parallel. This means that our work can be scaled up to large parallel computation frameworks. We present an efficient loss function for the discriminator, an effective adjustment component, and a suitable generator. We also show how to introduce the decay factor to stabilize the training procedure. We have performed extensive experiments on synthetic datasets, MNIST, and CIFAR-10. These experiments reveal that the error provided by the adjustment component could successfully separate the generated distributions and each of the generators can stably learn a part of the real distribution even if only a few modes are contained in the real distribution. Generative Adversarial Networks were proposed by BID7 , where two neural networks, generator and discriminator, are trained to play a minimax game. The generator is trained to fool the discriminator while the discriminator is trained to distinguish fake data (generated data) from real data. When Nash Equilibrium is reached, generated distribution P G will be equal to the real distribution P real . Unlike Restricted Boltzmann Machine (RBM, BID20 or Variational Auto-encoder (VAE, BID11 ), that explicitly approximate data distribution, the approximation of GAN is implicit. Due to this property, training GAN is challenging. It has been reported that GAN suffers from the mode collapse problem BID6 , BID14 ). Many methods have been proposed to solve this problem BID21 , BID22 , BID8 , ). In this paper, we propose a new model to solve this problem.Similar to the work of BID18 ), we use a set of generators to replace the single, complex generator. Each generator only captures a part of the real distribution, while the distance between the mix-generated distribution and the real distribution should be minimized. An adjustment component is added to achieve separation between each pair of generators, and a penalty will be passed to the generator if an overlap is detected. Moreover, we propose a shrinkage adjustment component method to gradually reduce the effect of the penalty, since the strict boundary will lead to a nonconvergence problem. Practically, forcing each generated distribution to be totally disjoint will cause potential problems. More specifically, we observe two problems in practice: (1) competition: multiple generators try to capture one mode, but are hampered by a strict boundary. This happens when the total number of generators K is greater than the actual number of modes of P real . (2) One beats all: One or a few of the generators are strong enough to capture all the modes, while the other generators are blocked outside and capture nothing. To solve these problems, we propose the following approach: (1) use reverse KL divergence instead of JS Divergence as the generator loss, to reduce the generator's ability to capture all the modes, and (2) introduce a shrinkage adjustment method to gradually reduce the weight of the adjustment component C based on the training time and the difference between each generator loss. We will discuss the details in part 3. Benefiting from such design, there is no need to pre-define the number of generators, and stable convergence can be obtained when the new component shrinks to zero. Finally, our model can allow parallelized training among generators, with synchronized or asynchronized updated for the discriminator, which reduces the training time.To highlight, our main contributions are:1. In Sections 3.1 and 2, we propose a multi-generator model where each generator captures different parts of the real data distribution while the mixing distribution captures all the data.2. We introduce an adjustment component to separate between generated distributions. The adjustment can work with any discriminator.3. In Section 3.3, we propose a shrinkage component method which reduces the penalty to guarantee convergence. If the penalty shrinks to zero, we will minimize DISPLAYFORM0 We organize the shared memory to allow for parallel training to reduce the training time.Our algorithm scales well even on large parallel platforms.5. In Section 4, we use synthetic and real data to illustrate the effectiveness of our design. In this paper, we propose a mixed generator method to solve the mode collapse problem of GAN, and our algorithm is parallelizable, and can be scaled to large platforms. To conquer the competition and one-beat all problems in the mix generator model, we have designed the reverse KL divergence loss function, and an adjustment component decay to produce a stable, converging, and fast training method. The results show we can handle the situation when the generators compete for the same mode even when the number of generators is greater than the number of modes. The shrinkage method which gradually reduced extra component to zero will eliminate the adjustment player and reduce to multi-generator vs discriminator game.More works need to be done in this multi-player game. First, the shrinkage method can also be improved if we can have a better heuristic for β. Or we can train to learn β, to achieve balance between competition and convergence. Second, the weight for each generator can also be dynamic. The generator learns more should have higher weight. Finally, new parallelization algorithm with less communication cost could be investigate to accelerate the multi-generator model since currently the run time is far from optimal.
multi generator to capture Pdata, solve the competition and one-beat-all problem Proposes parallel GANs to avoid mode collapse in GANs through a combination of multiple weak generators. 