Second-Order Uncertainty Quantification: Variance-Based Measures
Yusuf Sale Paul Hofman Lisa Wimmer Eyke Hüllermeier Thomas Nagler
Uncertainty quantification is a critical aspect of machine learning models,
providing important insights into the reliability of predictions and aiding the
decision-making process in real-world applications. This paper proposes a novel
way to use variance-based measures to quantify uncertainty on the basis of
second-order distributions in classification problems. A distinctive feature of
the measures is the ability to reason about uncertainties on a class-based
level, which is useful in situations where nuanced decision-making is required.
Recalling some properties from the literature, we highlight that the
variance-based measures satisfy important (axiomatic) properties. In addition
to this axiomatic approach, we present empirical results showing the measures
to be effective and competitive to commonly used entropy-based measures.