Classification of Data Generated by Gaussian Mixture Models Using Deep
  ReLU Networks
Tian-Yi Zhou Xiaoming Huo
This paper studies the binary classification of unbounded data from ${\mathbb
R}^d$ generated under Gaussian Mixture Models (GMMs) using deep ReLU neural
networks. We obtain $\unicode{x2013}$ for the first time $\unicode{x2013}$
non-asymptotic upper bounds and convergence rates of the excess risk (excess
misclassification error) for the classification without restrictions on model
parameters. The convergence rates we derive do not depend on dimension $d$,
demonstrating that deep ReLU networks can overcome the curse of dimensionality
in classification. While the majority of existing generalization analysis of
classification algorithms relies on a bounded domain, we consider an unbounded
domain by leveraging the analyticity and fast decay of Gaussian distributions.
To facilitate our analysis, we give a novel approximation error bound for
general analytic functions using ReLU networks, which may be of independent
interest. Gaussian distributions can be adopted nicely to model data arising in
applications, e.g., speeches, images, and texts; our results provide a
theoretical verification of the observed efficiency of deep neural networks in
practical classification problems.