Anomaly Detection by Adapting a pre-trained Vision Language Model
Yuxuan Cai Xinwei He Dingkang Liang Ao Tong Xiang Bai
Recently, large vision and language models have shown their success when
adapting them to many downstream tasks. In this paper, we present a unified
framework named CLIP-ADA for Anomaly Detection by Adapting a pre-trained CLIP
model. To this end, we make two important improvements: 1) To acquire unified
anomaly detection across industrial images of multiple categories, we introduce
the learnable prompt and propose to associate it with abnormal patterns through
self-supervised learning. 2) To fully exploit the representation power of CLIP,
we introduce an anomaly region refinement strategy to refine the localization
quality. During testing, the anomalies are localized by directly calculating
the similarity between the representation of the learnable prompt and the
image. Comprehensive experiments demonstrate the superiority of our framework,
e.g., we achieve the state-of-the-art 97.5/55.6 and 89.3/33.1 on MVTec-AD and
VisA for anomaly detection and localization. In addition, the proposed method
also achieves encouraging performance with marginal training data, which is
more challenging.