Stylist: Style-Driven Feature Ranking for Robust Novelty Detection
Stefan Smeu Elena Burceanu Emanuela Haller Andrei Liviu Nicolicioiu
Novelty detection aims at finding samples that differ in some form from the
distribution of seen samples. But not all changes are created equal. Data can
suffer a multitude of distribution shifts, and we might want to detect only
some types of relevant changes. Similar to works in out-of-distribution
generalization, we propose to use the formalization of separating into semantic
or content changes, that are relevant to our task, and style changes, that are
irrelevant. Within this formalization, we define the robust novelty detection
as the task of finding semantic changes while being robust to style
distributional shifts. Leveraging pretrained, large-scale model
representations, we introduce Stylist, a novel method that focuses on dropping
environment-biased features. First, we compute a per-feature score based on the
feature distribution distances between environments. Next, we show that our
selection manages to remove features responsible for spurious correlations and
improve novelty detection performance. For evaluation, we adapt domain
generalization datasets to our task and analyze the methods behaviors. We
additionally built a large synthetic dataset where we have control over the
spurious correlations degree. We prove that our selection mechanism improves
novelty detection algorithms across multiple datasets, containing both
stylistic and content shifts.