Training Implicit Generative Models via an Invariant Statistical Loss
José Manuel de Frutos Pablo M. Olmos Manuel A. Vázquez Joaquín Míguez
Implicit generative models have the capability to learn arbitrary complex
data distributions. On the downside, training requires telling apart real data
from artificially-generated ones using adversarial discriminators, leading to
unstable training and mode-dropping issues. As reported by Zahee et al. (2017),
even in the one-dimensional (1D) case, training a generative adversarial
network (GAN) is challenging and often suboptimal. In this work, we develop a
discriminator-free method for training one-dimensional (1D) generative implicit
models and subsequently expand this method to accommodate multivariate cases.
Our loss function is a discrepancy measure between a suitably chosen
transformation of the model samples and a uniform distribution; hence, it is
invariant with respect to the true distribution of the data. We first formulate
our method for 1D random variables, providing an effective solution for
approximate reparameterization of arbitrary complex distributions. Then, we
consider the temporal setting (both univariate and multivariate), in which we
model the conditional distribution of each sample given the history of the
process. We demonstrate through numerical simulations that this new method
yields promising results, successfully learning true distributions in a variety
of scenarios and mitigating some of the well-known problems that
state-of-the-art implicit methods present.