A Gauss-Newton Approach for Min-Max Optimization in Generative
  Adversarial Networks
Neel Mishra Bamdev Mishra Pratik Jawanpuria Pawan Kumar
A novel first-order method is proposed for training generative adversarial
networks (GANs). It modifies the Gauss-Newton method to approximate the min-max
Hessian and uses the Sherman-Morrison inversion formula to calculate the
inverse. The method corresponds to a fixed-point method that ensures necessary
contraction. To evaluate its effectiveness, numerical experiments are conducted
on various datasets commonly used in image generation tasks, such as MNIST,
Fashion MNIST, CIFAR10, FFHQ, and LSUN. Our method is capable of generating
high-fidelity images with greater diversity across multiple datasets. It also
achieves the highest inception score for CIFAR10 among all compared methods,
including state-of-the-art second-order methods. Additionally, its execution
time is comparable to that of first-order min-max methods.