Accelerating Ensemble Error Bar Prediction with Single Models Fits
Vidit Agrawal Shixin Zhang Lane E. Schultz Dane Morgan
Ensemble models can be used to estimate prediction uncertainties in machine
learning models. However, an ensemble of N models is approximately N times more
computationally demanding compared to a single model when it is used for
inference. In this work, we explore fitting a single model to predicted
ensemble error bar data, which allows us to estimate uncertainties without the
need for a full ensemble. Our approach is based on three models: Model A for
predictive accuracy, Model $A_{E}$ for traditional ensemble-based error bar
prediction, and Model B, fit to data from Model $A_{E}$, to be used for
predicting the values of $A_{E}$ but with only one model evaluation. Model B
leverages synthetic data augmentation to estimate error bars efficiently. This
approach offers a highly flexible method of uncertainty quantification that can
approximate that of ensemble methods but only requires a single extra model
evaluation over Model A during inference. We assess this approach on a set of
problems in materials science.