Understanding the Feature Norm for Out-of-Distribution Detection
Jaewoo Park Jacky Chen Long Chai Jaeho Yoon Andrew Beng Jin Teoh
A neural network trained on a classification dataset often exhibits a higher
vector norm of hidden layer features for in-distribution (ID) samples, while
producing relatively lower norm values on unseen instances from
out-of-distribution (OOD). Despite this intriguing phenomenon being utilized in
many applications, the underlying cause has not been thoroughly investigated.
In this study, we demystify this very phenomenon by scrutinizing the
discriminative structures concealed in the intermediate layers of a neural
network. Our analysis leads to the following discoveries: (1) The feature norm
is a confidence value of a classifier hidden in the network layer, specifically
its maximum logit. Hence, the feature norm distinguishes OOD from ID in the
same manner that a classifier confidence does. (2) The feature norm is
class-agnostic, thus it can detect OOD samples across diverse discriminative
models. (3) The conventional feature norm fails to capture the deactivation
tendency of hidden layer neurons, which may lead to misidentification of ID
samples as OOD instances. To resolve this drawback, we propose a novel
negative-aware norm (NAN) that can capture both the activation and deactivation
tendencies of hidden layer neurons. We conduct extensive experiments on NAN,
demonstrating its efficacy and compatibility with existing OOD detectors, as
well as its capability in label-free environments.