TSONN: Time-stepping-oriented neural network for solving partial
  differential equations
Wenbo Cao Weiwei Zhang
Deep neural networks (DNNs), especially physics-informed neural networks
(PINNs), have recently become a new popular method for solving forward and
inverse problems governed by partial differential equations (PDEs). However,
these methods still face challenges in achieving stable training and obtaining
correct results in many problems, since minimizing PDE residuals with PDE-based
soft constraint make the problem ill-conditioned. Different from all existing
methods that directly minimize PDE residuals, this work integrates
time-stepping method with deep learning, and transforms the original
ill-conditioned optimization problem into a series of well-conditioned
sub-problems over given pseudo time intervals. The convergence of model
training is significantly improved by following the trajectory of the pseudo
time-stepping process, yielding a robust optimization-based PDE solver. Our
results show that the proposed method achieves stable training and correct
results in many problems that standard PINNs fail to solve, requiring only a
simple modification on the loss function. In addition, we demonstrate several
novel properties and advantages of time-stepping methods within the framework
of neural network-based optimization approach, in comparison to traditional
grid-based numerical method. Specifically, explicit scheme allows significantly
larger time step, while implicit scheme can be implemented as straightforwardly
as explicit scheme.