Continual Improvement of Threshold-Based Novelty Detection
Abe Ejilemele Jorge Mendez-Mendez
When evaluated in dynamic, open-world situations, neural networks struggle to
detect unseen classes. This issue complicates the deployment of continual
learners in realistic environments where agents are not explicitly informed
when novel categories are encountered. A common family of techniques for
detecting novelty relies on thresholds of similarity between observed data
points and the data used for training. However, these methods often require
manually specifying (ahead of time) the value of these thresholds, and are
therefore incapable of adapting to the nature of the data. We propose a new
method for automatically selecting these thresholds utilizing a linear search
and leave-one-out cross-validation on the ID classes. We demonstrate that this
novel method for selecting thresholds results in improved total accuracy on
MNIST, Fashion MNIST, and CIFAR-10.