Convergences for Minimax Optimization Problems over Infinite-Dimensional
  Spaces Towards Stability in Adversarial Training
Takashi Furuya Satoshi Okuda Kazuma Suetake Yoshihide Sawada
Training neural networks that require adversarial optimization, such as
generative adversarial networks (GANs) and unsupervised domain adaptations
(UDAs), suffers from instability. This instability problem comes from the
difficulty of the minimax optimization, and there have been various approaches
in GANs and UDAs to overcome this problem. In this study, we tackle this
problem theoretically through a functional analysis. Specifically, we show the
convergence property of the minimax problem by the gradient descent over the
infinite-dimensional spaces of continuous functions and probability measures
under certain conditions. Using this setting, we can discuss GANs and UDAs
comprehensively, which have been studied independently. In addition, we show
that the conditions necessary for the convergence property are interpreted as
stabilization techniques of adversarial training such as the spectral
normalization and the gradient penalty.