Model-Free Local Recalibration of Neural Networks
R. Torres D. J. Nott S. A. Sisson T. Rodrigues J. G. Reis G. S. Rodrigues
Artificial neural networks (ANNs) are highly flexible predictive models.
However, reliably quantifying uncertainty for their predictions is a continuing
challenge. There has been much recent work on "recalibration" of predictive
distributions for ANNs, so that forecast probabilities for events of interest
are consistent with certain frequency evaluations of them. Uncalibrated
probabilistic forecasts are of limited use for many important decision-making
tasks. To address this issue, we propose a localized recalibration of ANN
predictive distributions using the dimension-reduced representation of the
input provided by the ANN hidden layers. Our novel method draws inspiration
from recalibration techniques used in the literature on approximate Bayesian
computation and likelihood-free inference methods. Most existing calibration
methods for ANNs can be thought of as calibrating either on the input layer,
which is difficult when the input is high-dimensional, or the output layer,
which may not be sufficiently flexible. Through a simulation study, we
demonstrate that our method has good performance compared to alternative
approaches, and explore the benefits that can be achieved by localizing the
calibration based on different layers of the network. Finally, we apply our
proposed method to a diamond price prediction problem, demonstrating the
potential of our approach to improve prediction and uncertainty quantification
in real-world applications.