Solving multiscale elliptic problems by sparse radial basis function
  neural networks
Zhiwen Wang Minxin Chen Jingrun Chen
Machine learning has been successfully applied to various fields of
scientific computing in recent years. In this work, we propose a sparse radial
basis function neural network method to solve elliptic partial differential
equations (PDEs) with multiscale coefficients. Inspired by the deep mixed
residual method, we rewrite the second-order problem into a first-order system
and employ multiple radial basis function neural networks (RBFNNs) to
approximate unknown functions in the system. To aviod the overfitting due to
the simplicity of RBFNN, an additional regularization is introduced in the loss
function. Thus the loss function contains two parts: the $L_2$ loss for the
residual of the first-order system and boundary conditions, and the $\ell_1$
regularization term for the weights of radial basis functions (RBFs). An
algorithm for optimizing the specific loss function is introduced to accelerate
the training process. The accuracy and effectiveness of the proposed method are
demonstrated through a collection of multiscale problems with scale separation,
discontinuity and multiple scales from one to three dimensions. Notably, the
$\ell_1$ regularization can achieve the goal of representing the solution by
fewer RBFs. As a consequence, the total number of RBFs scales like
$\mathcal{O}(\varepsilon^{-n\tau})$, where $\varepsilon$ is the smallest scale,
$n$ is the dimensionality, and $\tau$ is typically smaller than $1$. It is
worth mentioning that the proposed method not only has the numerical
convergence and thus provides a reliable numerical solution in three dimensions
when a classical method is typically not affordable, but also outperforms most
other available machine learning methods in terms of accuracy and robustness.