Towards Optimal Feature-Shaping Methods for Out-of-Distribution
  Detection
Qinyu Zhao Ming Xu Kartik Gupta Akshay Asthana Liang Zheng Stephen Gould
Feature shaping refers to a family of methods that exhibit state-of-the-art
performance for out-of-distribution (OOD) detection. These approaches
manipulate the feature representation, typically from the penultimate layer of
a pre-trained deep learning model, so as to better differentiate between
in-distribution (ID) and OOD samples. However, existing feature-shaping methods
usually employ rules manually designed for specific model architectures and OOD
datasets, which consequently limit their generalization ability. To address
this gap, we first formulate an abstract optimization framework for studying
feature-shaping methods. We then propose a concrete reduction of the framework
with a simple piecewise constant shaping function and show that existing
feature-shaping methods approximate the optimal solution to the concrete
optimization problem. Further, assuming that OOD data is inaccessible, we
propose a formulation that yields a closed-form solution for the piecewise
constant shaping function, utilizing solely the ID data. Through extensive
experiments, we show that the feature-shaping function optimized by our method
improves the generalization ability of OOD detection across a large variety of
datasets and model architectures.