Salient Mask-Guided Vision Transformer for Fine-Grained Classification
Dmitry Demidov Muhammad Hamza Sharif Aliakbar Abdurahimov Hisham Cholakkal Fahad Shahbaz Khan
Fine-grained visual classification (FGVC) is a challenging computer vision
problem, where the task is to automatically recognise objects from subordinate
categories. One of its main difficulties is capturing the most discriminative
inter-class variances among visually similar classes. Recently, methods with
Vision Transformer (ViT) have demonstrated noticeable achievements in FGVC,
generally by employing the self-attention mechanism with additional
resource-consuming techniques to distinguish potentially discriminative regions
while disregarding the rest. However, such approaches may struggle to
effectively focus on truly discriminative regions due to only relying on the
inherent self-attention mechanism, resulting in the classification token likely
aggregating global information from less-important background patches.
Moreover, due to the immense lack of the datapoints, classifiers may fail to
find the most helpful inter-class distinguishing features, since other
unrelated but distinctive background regions may be falsely recognised as being
valuable. To this end, we introduce a simple yet effective Salient Mask-Guided
Vision Transformer (SM-ViT), where the discriminability of the standard ViT`s
attention maps is boosted through salient masking of potentially discriminative
foreground regions. Extensive experiments demonstrate that with the standard
training procedure our SM-ViT achieves state-of-the-art performance on popular
FGVC benchmarks among existing ViT-based approaches while requiring fewer
resources and lower input image resolution.