Favour: FAst Variance Operator for Uncertainty Rating
Thomas D. Ahle Sahar Karimi Peter Tak Peter Tang
Bayesian Neural Networks (BNN) have emerged as a crucial approach for
interpreting ML predictions. By sampling from the posterior distribution, data
scientists may estimate the uncertainty of an inference. Unfortunately many
inference samples are often needed, the overhead of which greatly hinder BNN's
wide adoption. To mitigate this, previous work proposed propagating the first
and second moments of the posterior directly through the network. However, on
its own this method is even slower than sampling, so the propagated variance
needs to be approximated such as assuming independence between neural nodes.
The resulting trade-off between quality and inference time did not match even
plain Monte Carlo sampling.
  Our contribution is a more principled variance propagation framework based on
"spiked covariance matrices", which smoothly interpolates between quality and
inference time. This is made possible by a new fast algorithm for updating a
diagonal-plus-low-rank matrix approximation under various operations. We tested
our algorithm against sampling based MC Dropout and Variational Inference on a
number of downstream uncertainty themed tasks, such as calibration and
out-of-distribution testing. We find that Favour is as fast as performing 2-3
inference samples, while matching the performance of 10-100 samples.
  In summary, this work enables the use of BNN in the realm of performance
critical tasks where they have previously been out of reach.