PROSE: Predicting Operators and Symbolic Expressions using Multimodal
  Transformers
Yuxuan Liu Zecheng Zhang Hayden Schaeffer
Approximating nonlinear differential equations using a neural network
provides a robust and efficient tool for various scientific computing tasks,
including real-time predictions, inverse problems, optimal controls, and
surrogate modeling. Previous works have focused on embedding dynamical systems
into networks through two approaches: learning a single solution operator
(i.e., the mapping from input parametrized functions to solutions) or learning
the governing system of equations (i.e., the constitutive model relative to the
state variables). Both of these approaches yield different representations for
the same underlying data or function. Additionally, observing that families of
differential equations often share key characteristics, we seek one network
representation across a wide range of equations. Our method, called Predicting
Operators and Symbolic Expressions (PROSE), learns maps from multimodal inputs
to multimodal outputs, capable of generating both numerical predictions and
mathematical equations. By using a transformer structure and a feature fusion
approach, our network can simultaneously embed sets of solution operators for
various parametric differential equations using a single trained network.
Detailed experiments demonstrate that the network benefits from its multimodal
nature, resulting in improved prediction accuracy and better generalization.
The network is shown to be able to handle noise in the data and errors in the
symbolic representation, including noisy numerical values, model
misspecification, and erroneous addition or deletion of terms. PROSE provides a
new neural network framework for differential equations which allows for more
flexibility and generality in learning operators and governing equations from
data.