Class-Discriminative Attention Maps for Vision Transformers
Lennart Brocki Neo Christopher Chung
Interpretability methods are critical components for examining and exploring
deep neural networks (DNN), as well as increasing our understanding of and
trust in them. Vision transformers (ViT), which can be trained to
state-of-the-art performance with a self-supervised learning (SSL) training
method, provide built-in attention maps (AM). While AMs can provide
high-quality semantic segmentation of input images, they do not account for any
signal coming from a downstream classifier. We introduce class-discriminative
attention maps (CDAM), a novel post-hoc explanation method that is highly
sensitive to the target class. Our method essentially scales attention scores
by how relevant the corresponding tokens are for the predictions of a
classifier head. Alternative to classifier outputs, CDAM can also explain a
user-defined concept by targeting similarity measures in the latent space of
the ViT. This allows for explanations of arbitrary concepts, defined by the
user through a few sample images. We investigate the operating characteristics
of CDAM in comparison with relevance propagation (RP) and token ablation maps
(TAM), an alternative to pixel occlusion methods. CDAM is highly
class-discriminative and semantically relevant, while providing implicit
regularization of relevance scores.
  PyTorch implementation: \url{https://github.com/lenbrocki/CDAM}
  Web live demo: \url{https://cdam.informatism.com/}