ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection
Bo Peng Yadan Luo Yonggang Zhang Yixuan Li Zhen Fang
Post-hoc out-of-distribution (OOD) detection has garnered intensive attention
in reliable machine learning. Many efforts have been dedicated to deriving
score functions based on logits, distances, or rigorous data distribution
assumptions to identify low-scoring OOD samples. Nevertheless, these estimate
scores may fail to accurately reflect the true data density or impose
impractical constraints. To provide a unified perspective on density-based
score design, we propose a novel theoretical framework grounded in Bregman
divergence, which extends distribution considerations to encompass an
exponential family of distributions. Leveraging the conjugation constraint
revealed in our theorem, we introduce a \textsc{ConjNorm} method, reframing
density function design as a search for the optimal norm coefficient $p$
against the given dataset. In light of the computational challenges of
normalization, we devise an unbiased and analytically tractable estimator of
the partition function using the Monte Carlo-based importance sampling
technique. Extensive experiments across OOD detection benchmarks empirically
demonstrate that our proposed \textsc{ConjNorm} has established a new
state-of-the-art in a variety of OOD detection setups, outperforming the
current best method by up to 13.25$\%$ and 28.19$\%$ (FPR95) on CIFAR-100 and
ImageNet-1K, respectively.