DOF: Accelerating High-order Differential Operators with Forward
  Propagation
Ruichen Li Chuwei Wang Haotian Ye Di He Liwei Wang
Solving partial differential equations (PDEs) efficiently is essential for
analyzing complex physical systems. Recent advancements in leveraging deep
learning for solving PDE have shown significant promise. However, machine
learning methods, such as Physics-Informed Neural Networks (PINN), face
challenges in handling high-order derivatives of neural network-parameterized
functions. Inspired by Forward Laplacian, a recent method of accelerating
Laplacian computation, we propose an efficient computational framework,
Differential Operator with Forward-propagation (DOF), for calculating general
second-order differential operators without losing any precision. We provide
rigorous proof of the advantages of our method over existing methods,
demonstrating two times improvement in efficiency and reduced memory
consumption on any architectures. Empirical results illustrate that our method
surpasses traditional automatic differentiation (AutoDiff) techniques,
achieving 2x improvement on the MLP structure and nearly 20x improvement on the
MLP with Jacobian sparsity.