A SAM-guided Two-stream Lightweight Model for Anomaly Detection
Chenghao Li Lei Qi Xin Geng
In industrial anomaly detection, model efficiency and mobile-friendliness
become the primary concerns in real-world applications. Simultaneously, the
impressive generalization capabilities of Segment Anything (SAM) have garnered
broad academic attention, making it an ideal choice for localizing unseen
anomalies and diverse real-world patterns. In this paper, considering these two
critical factors, we propose a SAM-guided Two-stream Lightweight Model for
unsupervised anomaly detection (STLM) that not only aligns with the two
practical application requirements but also harnesses the robust generalization
capabilities of SAM. We employ two lightweight image encoders, i.e., our
two-stream lightweight module, guided by SAM's knowledge. To be specific, one
stream is trained to generate discriminative and general feature
representations in both normal and anomalous regions, while the other stream
reconstructs the same images without anomalies, which effectively enhances the
differentiation of two-stream representations when facing anomalous regions.
Furthermore, we employ a shared mask decoder and a feature aggregation module
to generate anomaly maps. Our experiments conducted on MVTec AD benchmark show
that STLM, with about 16M parameters and achieving an inference time in 20ms,
competes effectively with state-of-the-art methods in terms of performance,
98.26% on pixel-level AUC and 94.92% on PRO. We further experiment on more
difficult datasets, e.g., VisA and DAGM, to demonstrate the effectiveness and
generalizability of STLM.