Sup-Norm Convergence of Deep Neural Network Estimator for Nonparametric
  Regression by Adversarial Training
Masaaki Imaizumi
We show the sup-norm convergence of deep neural network estimators with a
novel adversarial training scheme. For the nonparametric regression problem, it
has been shown that an estimator using deep neural networks can achieve better
performances in the sense of the $L2$-norm. In contrast, it is difficult for
the neural estimator with least-squares to achieve the sup-norm convergence,
due to the deep structure of neural network models. In this study, we develop
an adversarial training scheme and investigate the sup-norm convergence of deep
neural network estimators. First, we find that ordinary adversarial training
makes neural estimators inconsistent. Second, we show that a deep neural
network estimator achieves the optimal rate in the sup-norm sense by the
proposed adversarial training with correction. We extend our adversarial
training to general setups of a loss function and a data-generating function.
Our experiments support the theoretical findings.