Quantification of Predictive Uncertainty via Inference-Time Sampling
Katarína Tóthová Ľubor Ladický Daniel Thul Marc Pollefeys Ender Konukoglu
Predictive variability due to data ambiguities has typically been addressed
via construction of dedicated models with built-in probabilistic capabilities
that are trained to predict uncertainty estimates as variables of interest.
These approaches require distinct architectural components and training
mechanisms, may include restrictive assumptions and exhibit overconfidence,
i.e., high confidence in imprecise predictions. In this work, we propose a
post-hoc sampling strategy for estimating predictive uncertainty accounting for
data ambiguity. The method can generate different plausible outputs for a given
input and does not assume parametric forms of predictive distributions. It is
architecture agnostic and can be applied to any feed-forward deterministic
network without changes to the architecture or training procedure. Experiments
on regression tasks on imaging and non-imaging input data show the method's
ability to generate diverse and multi-modal predictive distributions, and a
desirable correlation of the estimated uncertainty with the prediction error.