GradOrth: A Simple yet Efficient Out-of-Distribution Detection with
  Orthogonal Projection of Gradients
Sima Behpour Thang Doan Xin Li Wenbin He Liang Gou Liu Ren
Detecting out-of-distribution (OOD) data is crucial for ensuring the safe
deployment of machine learning models in real-world applications. However,
existing OOD detection approaches primarily rely on the feature maps or the
full gradient space information to derive OOD scores neglecting the role of
most important parameters of the pre-trained network over in-distribution (ID)
data. In this study, we propose a novel approach called GradOrth to facilitate
OOD detection based on one intriguing observation that the important features
to identify OOD data lie in the lower-rank subspace of in-distribution (ID)
data. In particular, we identify OOD data by computing the norm of gradient
projection on the subspaces considered important for the in-distribution data.
A large orthogonal projection value (i.e. a small projection value) indicates
the sample as OOD as it captures a weak correlation of the ID data. This simple
yet effective method exhibits outstanding performance, showcasing a notable
reduction in the average false positive rate at a 95% true positive rate
(FPR95) of up to 8% when compared to the current state-of-the-art methods.