Unveiling Empirical Pathologies of Laplace Approximation for Uncertainty
  Estimation
Maksim Zhdanov Stanislav Dereka Sergey Kolesnikov
In this paper, we critically evaluate Bayesian methods for uncertainty
estimation in deep learning, focusing on the widely applied Laplace
approximation and its variants. Our findings reveal that the conventional
method of fitting the Hessian matrix negatively impacts out-of-distribution
(OOD) detection efficiency. We propose a different point of view, asserting
that focusing solely on optimizing prior precision can yield more accurate
uncertainty estimates in OOD detection while preserving adequate calibration
metrics. Moreover, we demonstrate that this property is not connected to the
training stage of a model but rather to its intrinsic properties. Through
extensive experimental evaluation, we establish the superiority of our
simplified approach over traditional methods in the out-of-distribution domain.