Weighted variation spaces and approximation by shallow ReLU networks
Ronald DeVore Robert D. Nowak Rahul Parhi Jonathan W. Siegel
We investigate the approximation of functions $f$ on a bounded domain
$\Omega\subset \mathbb{R}^d$ by the outputs of single-hidden-layer ReLU neural
networks of width $n$. This form of nonlinear $n$-term dictionary approximation
has been intensely studied since it is the simplest case of neural network
approximation (NNA). There are several celebrated approximation results for
this form of NNA that introduce novel model classes of functions on $\Omega$
whose approximation rates avoid the curse of dimensionality. These novel
classes include Barron classes, and classes based on sparsity or variation such
as the Radon-domain BV classes.
  The present paper is concerned with the definition of these novel model
classes on domains $\Omega$. The current definition of these model classes does
not depend on the domain $\Omega$. A new and more proper definition of model
classes on domains is given by introducing the concept of weighted variation
spaces. These new model classes are intrinsic to the domain itself. The
importance of these new model classes is that they are strictly larger than the
classical (domain-independent) classes. Yet, it is shown that they maintain the
same NNA rates.