Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling
Ruijia Niu Dongxia Wu Kai Kim Yi-An Ma Duncan Watson-Parris Rose Yu
Multi-fidelity surrogate modeling aims to learn an accurate surrogate at the
highest fidelity level by combining data from multiple sources. Traditional
methods relying on Gaussian processes can hardly scale to high-dimensional
data. Deep learning approaches utilize neural network based encoders and
decoders to improve scalability. These approaches share encoded representations
across fidelities without including corresponding decoder parameters. At the
highest fidelity, the representations are decoded with different parameters,
making the shared information inherently inaccurate. This hinders inference
performance, especially in out-of-distribution scenarios when the highest
fidelity data has limited domain coverage. To address these limitations, we
propose Multi-fidelity Residual Neural Processes (MFRNP), a novel
multi-fidelity surrogate modeling framework. MFRNP optimizes lower fidelity
decoders for accurate information sharing by aggregating lower fidelity
surrogate outputs and models residual between the aggregation and ground truth
on the highest fidelity. We show that MFRNP significantly outperforms current
state-of-the-art in learning partial differential equations and a real-world
climate modeling task.