SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection
  with Multimodal Large Language Models
Yichen Shi Yuhao Gao Yingxin Lai Hongyang Wang Jun Feng Lei He Jun Wan Changsheng Chen Zitong Yu Xiaochun Cao
Multimodal large language models (MLLMs) have demonstrated remarkable
problem-solving capabilities in various vision fields (e.g., generic object
recognition and grounding) based on strong visual semantic representation and
language reasoning ability. However, whether MLLMs are sensitive to subtle
visual spoof/forged clues and how they perform in the domain of face attack
detection (e.g., face spoofing and forgery detection) is still unexplored. In
this paper, we introduce a new benchmark, namely SHIELD, to evaluate the
ability of MLLMs on face spoofing and forgery detection. Specifically, we
design true/false and multiple-choice questions to evaluate multimodal face
data in these two face security tasks. For the face anti-spoofing task, we
evaluate three different modalities (i.e., RGB, infrared, depth) under four
types of presentation attacks (i.e., print attack, replay attack, rigid mask,
paper mask). For the face forgery detection task, we evaluate GAN-based and
diffusion-based data with both visual and acoustic modalities. Each question is
subjected to both zero-shot and few-shot tests under standard and chain of
thought (COT) settings. The results indicate that MLLMs hold substantial
potential in the face security domain, offering advantages over traditional
specific models in terms of interpretability, multimodal flexible reasoning,
and joint face spoof and forgery detection. Additionally, we develop a novel
Multi-Attribute Chain of Thought (MA-COT) paradigm for describing and judging
various task-specific and task-irrelevant attributes of face images, which
provides rich task-related knowledge for subtle spoof/forged clue mining.
Extensive experiments in separate face anti-spoofing, separate face forgery
detection, and joint detection tasks demonstrate the effectiveness of the
proposed MA-COT. The project is available at
https$:$//github.com/laiyingxin2/SHIELD