COOD: Combined out-of-distribution detection using multiple measures for
  anomaly & novel class detection in large-scale hierarchical classification
L. E. Hogeweg R. Gangireddy D. Brunink V. J. Kalkman L. Cornelissen J. W. Kamminga
High-performing out-of-distribution (OOD) detection, both anomaly and novel
class, is an important prerequisite for the practical use of classification
models. In this paper, we focus on the species recognition task in images
concerned with large databases, a large number of fine-grained hierarchical
classes, severe class imbalance, and varying image quality. We propose a
framework for combining individual OOD measures into one combined OOD (COOD)
measure using a supervised model. The individual measures are several existing
state-of-the-art measures and several novel OOD measures developed with novel
class detection and hierarchical class structure in mind. COOD was extensively
evaluated on three large-scale (500k+ images) biodiversity datasets in the
context of anomaly and novel class detection. We show that COOD outperforms
individual, including state-of-the-art, OOD measures by a large margin in terms
of TPR@1% FPR in the majority of experiments, e.g., improving detecting
ImageNet images (OOD) from 54.3% to 85.4% for the iNaturalist 2018 dataset.
SHAP (feature contribution) analysis shows that different individual OOD
measures are essential for various tasks, indicating that multiple OOD measures
and combinations are needed to generalize. Additionally, we show that
explicitly considering ID images that are incorrectly classified for the
original (species) recognition task is important for constructing
high-performing OOD detection methods and for practical applicability. The
framework can easily be extended or adapted to other tasks and media
modalities.