Building Flexible Machine Learning Models for Scientific Computing at
  Scale
Tianyu Chen Haoyi Zhou Ying Li Hao Wang Chonghan Gao Shanghang Zhang Jianxin Li
Foundation models have revolutionized knowledge acquisition across domains,
and our study introduces OmniArch, a paradigm-shifting approach designed for
building foundation models in multi-physics scientific computing. OmniArch's
pre-training involves a versatile pipeline that processes multi-physics
spatio-temporal data, casting forward problem learning into scalable
auto-regressive tasks, while our novel Physics-Informed Reinforcement Learning
(PIRL) technique during fine-tuning ensures alignment with physical laws.
Pre-trained on the comprehensive PDEBench dataset, OmniArch not only sets new
performance benchmarks for 1D, 2D and 3D PDEs but also demonstrates exceptional
adaptability to new physics via few-shot and zero-shot learning approaches. The
model's representations further extend to inverse problem-solving, highlighting
the transformative potential of AI-enabled Scientific Computing(AI4SC)
foundation models for engineering applications and physics discovery.