Deep neural networks with ReLU, leaky ReLU, and softplus activation
  provably overcome the curse of dimensionality for Kolmogorov partial
  differential equations with Lipschitz nonlinearities in the $L^p$-sense
Julia Ackermann Arnulf Jentzen Thomas Kruse Benno Kuckuck Joshua Lee Padgett
Recently, several deep learning (DL) methods for approximating
high-dimensional partial differential equations (PDEs) have been proposed. The
interest that these methods have generated in the literature is in large part
due to simulations which appear to demonstrate that such DL methods have the
capacity to overcome the curse of dimensionality (COD) for PDEs in the sense
that the number of computational operations they require to achieve a certain
approximation accuracy $\varepsilon\in(0,\infty)$ grows at most polynomially in
the PDE dimension $d\in\mathbb N$ and the reciprocal of $\varepsilon$. While
there is thus far no mathematical result that proves that one of such methods
is indeed capable of overcoming the COD, there are now a number of rigorous
results in the literature that show that deep neural networks (DNNs) have the
expressive power to approximate PDE solutions without the COD in the sense that
the number of parameters used to describe the approximating DNN grows at most
polynomially in both the PDE dimension $d\in\mathbb N$ and the reciprocal of
the approximation accuracy $\varepsilon>0$. Roughly speaking, in the literature
it is has been proved for every $T>0$ that solutions $u_d\colon
[0,T]\times\mathbb R^d\to \mathbb R$, $d\in\mathbb N$, of semilinear heat PDEs
with Lipschitz continuous nonlinearities can be approximated by DNNs with ReLU
activation at the terminal time in the $L^2$-sense without the COD provided
that the initial value functions $\mathbb R^d\ni x\mapsto u_d(0,x)\in\mathbb
R$, $d\in\mathbb N$, can be approximated by ReLU DNNs without the COD. It is
the key contribution of this work to generalize this result by establishing
this statement in the $L^p$-sense with $p\in(0,\infty)$ and by allowing the
activation function to be more general covering the ReLU, the leaky ReLU, and
the softplus activation functions as special cases.