The Anatomy of Adversarial Attacks: Concept-based XAI Dissection
Georgii Mikriukov Gesina Schwalbe Franz Motzkus Korinna Bade
Adversarial attacks (AAs) pose a significant threat to the reliability and
robustness of deep neural networks. While the impact of these attacks on model
predictions has been extensively studied, their effect on the learned
representations and concepts within these models remains largely unexplored. In
this work, we perform an in-depth analysis of the influence of AAs on the
concepts learned by convolutional neural networks (CNNs) using eXplainable
artificial intelligence (XAI) techniques. Through an extensive set of
experiments across various network architectures and targeted AA techniques, we
unveil several key findings. First, AAs induce substantial alterations in the
concept composition within the feature space, introducing new concepts or
modifying existing ones. Second, the adversarial perturbation itself can be
linearly decomposed into a set of latent vector components, with a subset of
these being responsible for the attack's success. Notably, we discover that
these components are target-specific, i.e., are similar for a given target
class throughout different AA techniques and starting classes. Our findings
provide valuable insights into the nature of AAs and their impact on learned
representations, paving the way for the development of more robust and
interpretable deep learning models, as well as effective defenses against
adversarial threats.