Video Anomaly Detection and Explanation via Large Language Models
Hui Lv Qianru Sun
Video Anomaly Detection (VAD) aims to localize abnormal events on the
timeline of long-range surveillance videos. Anomaly-scoring-based methods have
been prevailing for years but suffer from the high complexity of thresholding
and low explanability of detection results. In this paper, we conduct pioneer
research on equipping video-based large language models (VLLMs) in the
framework of VAD, making the VAD model free from thresholds and able to explain
the reasons for the detected anomalies. We introduce a novel network module
Long-Term Context (LTC) to mitigate the incapability of VLLMs in long-range
context modeling. We design a three-phase training method to improve the
efficiency of fine-tuning VLLMs by substantially minimizing the requirements
for VAD data and lowering the costs of annotating instruction-tuning data. Our
trained model achieves the top performance on the anomaly videos of the
UCF-Crime and TAD benchmarks, with the AUC improvements of +3.86\% and +4.96\%,
respectively. More impressively, our approach can provide textual explanations
for detected anomalies.