An Ambiguity Measure for Recognizing the Unknowns in Deep Learning
Roozbeh Yousefzadeh
We study the understanding of deep neural networks from the scope in which
they are trained on. While the accuracy of these models is usually impressive
on the aggregate level, they still make mistakes, sometimes on cases that
appear to be trivial. Moreover, these models are not reliable in realizing what
they do not know leading to failures such as adversarial vulnerability and
out-of-distribution failures. Here, we propose a measure for quantifying the
ambiguity of inputs for any given model with regard to the scope of its
training. We define the ambiguity based on the geometric arrangements of the
decision boundaries and the convex hull of training set in the feature space
learned by the trained model, and demonstrate that a single ambiguity measure
may detect a considerable portion of mistakes of a model on in-distribution
samples, adversarial inputs, as well as out-of-distribution inputs. Using our
ambiguity measure, a model may abstain from classification when it encounters
ambiguous inputs leading to a better model accuracy not just on a given testing
set, but on the inputs it may encounter at the world at large. In pursuit of
this measure, we develop a theoretical framework that can identify the unknowns
of the model in relation to its scope. We put this in perspective with the
confidence of the model and develop formulations to identify the regions of the
domain which are unknown to the model, yet the model is guaranteed to have high
confidence.