FM-ViT: Flexible Modal Vision Transformers for Face Anti-Spoofing
Ajian Liu Zichang Tan Zitong Yu Chenxu Zhao Jun Wan Yanyan Liang Zhen Lei Du Zhang Stan Z. Li Guodong Guo
The availability of handy multi-modal (i.e., RGB-D) sensors has brought about
a surge of face anti-spoofing research. However, the current multi-modal face
presentation attack detection (PAD) has two defects: (1) The framework based on
multi-modal fusion requires providing modalities consistent with the training
input, which seriously limits the deployment scenario. (2) The performance of
ConvNet-based model on high fidelity datasets is increasingly limited. In this
work, we present a pure transformer-based framework, dubbed the Flexible Modal
Vision Transformer (FM-ViT), for face anti-spoofing to flexibly target any
single-modal (i.e., RGB) attack scenarios with the help of available
multi-modal data. Specifically, FM-ViT retains a specific branch for each
modality to capture different modal information and introduces the Cross-Modal
Transformer Block (CMTB), which consists of two cascaded attentions named
Multi-headed Mutual-Attention (MMA) and Fusion-Attention (MFA) to guide each
modal branch to mine potential features from informative patch tokens, and to
learn modality-agnostic liveness features by enriching the modal information of
own CLS token, respectively. Experiments demonstrate that the single model
trained based on FM-ViT can not only flexibly evaluate different modal samples,
but also outperforms existing single-modal frameworks by a large margin, and
approaches the multi-modal frameworks introduced with smaller FLOPs and model
parameters.