Improving Vision Anomaly Detection with the Guidance of Language
  Modality
Dong Chen Kaihang Pan Guoming Wang Yueting Zhuang Siliang Tang
Recent years have seen a surge of interest in anomaly detection for tackling
industrial defect detection, event detection, etc. However, existing
unsupervised anomaly detectors, particularly those for the vision modality,
face significant challenges due to redundant information and sparse latent
space. Conversely, the language modality performs well due to its relatively
single data. This paper tackles the aforementioned challenges for vision
modality from a multimodal point of view. Specifically, we propose Cross-modal
Guidance (CMG), which consists of Cross-modal Entropy Reduction (CMER) and
Cross-modal Linear Embedding (CMLE), to tackle the redundant information issue
and sparse space issue, respectively. CMER masks parts of the raw image and
computes the matching score with the text. Then, CMER discards irrelevant
pixels to make the detector focus on critical contents. To learn a more compact
latent space for the vision anomaly detector, CMLE learns a correlation
structure matrix from the language modality, and then the latent space of
vision modality will be learned with the guidance of the matrix. Thereafter,
the vision latent space will get semantically similar images closer. Extensive
experiments demonstrate the effectiveness of the proposed methods.
Particularly, CMG outperforms the baseline that only uses images by 16.81%.
Ablation experiments further confirm the synergy among the proposed methods, as
each component depends on the other to achieve optimal performance.