T-TAME: Trainable Attention Mechanism for Explaining Convolutional
  Networks and Vision Transformers
Mariano V. Ntrougkas Nikolaos Gkalelis Vasileios Mezaris
The development and adoption of Vision Transformers and other deep-learning
architectures for image classification tasks has been rapid. However, the
"black box" nature of neural networks is a barrier to adoption in applications
where explainability is essential. While some techniques for generating
explanations have been proposed, primarily for Convolutional Neural Networks,
adapting such techniques to the new paradigm of Vision Transformers is
non-trivial. This paper presents T-TAME, Transformer-compatible Trainable
Attention Mechanism for Explanations, a general methodology for explaining deep
neural networks used in image classification tasks. The proposed architecture
and training technique can be easily applied to any convolutional or Vision
Transformer-like neural network, using a streamlined training approach. After
training, explanation maps can be computed in a single forward pass; these
explanation maps are comparable to or outperform the outputs of computationally
expensive perturbation-based explainability techniques, achieving SOTA
performance. We apply T-TAME to three popular deep learning classifier
architectures, VGG-16, ResNet-50, and ViT-B-16, trained on the ImageNet
dataset, and we demonstrate improvements over existing state-of-the-art
explainability methods. A detailed analysis of the results and an ablation
study provide insights into how the T-TAME design choices affect the quality of
the generated explanation maps.