OOD Aware Supervised Contrastive Learning
Soroush Seifi Daniel Olmeda Reino Nikolay Chumerin Rahaf Aljundi
Out-of-Distribution (OOD) detection is a crucial problem for the safe
deployment of machine learning models identifying samples that fall outside of
the training distribution, i.e. in-distribution data (ID). Most OOD works focus
on the classification models trained with Cross Entropy (CE) and attempt to fix
its inherent issues. In this work we leverage powerful representation learned
with Supervised Contrastive (SupCon) training and propose a holistic approach
to learn a classifier robust to OOD data. We extend SupCon loss with two
additional contrast terms. The first term pushes auxiliary OOD representations
away from ID representations without imposing any constraints on similarities
among auxiliary data. The second term pushes OOD features far from the existing
class prototypes, while pushing ID representations closer to their
corresponding class prototype. When auxiliary OOD data is not available, we
propose feature mixing techniques to efficiently generate pseudo-OOD features.
Our solution is simple and efficient and acts as a natural extension of the
closed-set supervised contrastive representation learning. We compare against
different OOD detection methods on the common benchmarks and show
state-of-the-art results.