Learn Suspected Anomalies from Event Prompts for Video Anomaly Detection
Chenchen Tao Chong Wang Yuexian Zou Xiaohao Peng Jiafei Wu Jiangbo Qian
Most models for weakly supervised video anomaly detection (WS-VAD) rely on
multiple instance learning, aiming to distinguish normal and abnormal snippets
without specifying the type of anomaly. The ambiguous nature of anomaly
definitions across contexts introduces bias in detecting abnormal and normal
snippets within the abnormal bag. Taking the first step to show the model why
it is anomalous, a novel framework is proposed to guide the learning of
suspected anomalies from event prompts. Given a textual prompt dictionary of
potential anomaly events and the captions generated from anomaly videos, the
semantic anomaly similarity between them could be calculated to identify the
suspected anomalous events for each video snippet. It enables a new
multi-prompt learning process to constrain the visual-semantic features across
all videos, as well as provides a new way to label pseudo anomalies for
self-training. To demonstrate effectiveness, comprehensive experiments and
detailed ablation studies are conducted on four datasets, namely XD-Violence,
UCF-Crime, TAD, and ShanghaiTech. Our proposed model outperforms most
state-of-the-art methods in terms of AP or AUC (82.6\%, 87.7\%, 93.1\%, and
97.4\%). Furthermore, it shows promising performance in open-set and
cross-dataset cases.