Towards a Foundation Model for Partial Differential Equation:
  Multi-Operator Learning and Extrapolation
Jingmin Sun Yuxuan Liu Zecheng Zhang Hayden Schaeffer
Foundation models, such as large language models, have demonstrated success
in addressing various language and image processing tasks. In this work, we
introduce a multi-modal foundation model for scientific problems, named
PROSE-PDE. Our model, designed for bi-modality to bi-modality learning, is a
multi-operator learning approach which can predict future states of
spatiotemporal systems while concurrently learning the underlying governing
equations of the physical system. Specifically, we focus on multi-operator
learning by training distinct one-dimensional time-dependent nonlinear constant
coefficient partial differential equations, with potential applications to many
physical applications including physics, geology, and biology. More
importantly, we provide three extrapolation studies to demonstrate that
PROSE-PDE can generalize physical features through the robust training of
multiple operators and that the proposed model can extrapolate to predict PDE
solutions whose models or data were unseen during the training. Furthermore, we
show through systematic numerical experiments that the utilization of the
symbolic modality in our model effectively resolves the well-posedness problems
with training multiple operators and thus enhances our model's predictive
capabilities.