Multifidelity linear regression for scientific machine learning from
  scarce data
Elizabeth Qian Anirban Chaudhuri Dayoung Kang Vignesh Sella
Machine learning (ML) methods, which fit to data the parameters of a given
parameterized model class, have garnered significant interest as potential
methods for learning surrogate models for complex engineering systems for which
traditional simulation is expensive. However, in many scientific and
engineering settings, generating high-fidelity data on which to train ML models
is expensive, and the available budget for generating training data is limited.
ML models trained on the resulting scarce high-fidelity data have high variance
and are sensitive to vagaries of the training data set. We propose a new
multifidelity training approach for scientific machine learning that exploits
the scientific context where data of varying fidelities and costs are
available; for example high-fidelity data may be generated by an expensive
fully resolved physics simulation whereas lower-fidelity data may arise from a
cheaper model based on simplifying assumptions. We use the multifidelity data
to define new multifidelity Monte Carlo estimators for the unknown parameters
of linear regression models, and provide theoretical analyses that guarantee
the approach's accuracy and improved robustness to small training budgets.
Numerical results verify the theoretical analysis and demonstrate that
multifidelity learned models trained on scarce high-fidelity data and
additional low-fidelity data achieve order-of-magnitude lower model variance
than standard models trained on only high-fidelity data of comparable cost.
This illustrates that in the scarce data regime, our multifidelity training
strategy yields models with lower expected error than standard training
approaches.