ExCeL : Combined Extreme and Collective Logit Information for Enhancing
  Out-of-Distribution Detection
Naveen Karunanayake Suranga Seneviratne Sanjay Chawla
Deep learning models often exhibit overconfidence in predicting
out-of-distribution (OOD) data, underscoring the crucial role of OOD detection
in ensuring reliability in predictions. Among various OOD detection approaches,
post-hoc detectors have gained significant popularity, primarily due to their
ease of use and implementation. However, the effectiveness of most post-hoc OOD
detectors has been constrained as they rely solely either on extreme
information, such as the maximum logit, or on the collective information (i.e.,
information spanned across classes or training samples) embedded within the
output layer. In this paper, we propose ExCeL that combines both extreme and
collective information within the output layer for enhanced accuracy in OOD
detection. We leverage the logit of the top predicted class as the extreme
information (i.e., the maximum logit), while the collective information is
derived in a novel approach that involves assessing the likelihood of other
classes appearing in subsequent ranks across various training samples. Our idea
is motivated by the observation that, for in-distribution (ID) data, the
ranking of classes beyond the predicted class is more deterministic compared to
that in OOD data. Experiments conducted on CIFAR100 and ImageNet-200 datasets
demonstrate that ExCeL consistently is among the five top-performing methods
out of twenty-one existing post-hoc baselines when the joint performance on
near-OOD and far-OOD is considered (i.e., in terms of AUROC and FPR95).
Furthermore, ExCeL shows the best overall performance across both datasets,
unlike other baselines that work best on one dataset but has a performance drop
in the other.