Targeted Visualization of the Backbone of Encoder LLMs
Isaac Roberts Alexander Schulz Luca Hermes Barbara Hammer
Attention based Large Language Models (LLMs) are the state-of-the-art in
natural language processing (NLP). The two most common architectures are
encoders such as BERT, and decoders like the GPT models. Despite the success of
encoder models, on which we focus in this work, they also bear several risks,
including issues with bias or their susceptibility for adversarial attacks,
signifying the necessity for explainable AI to detect such issues. While there
does exist various local explainability methods focusing on the prediction of
single inputs, global methods based on dimensionality reduction for
classification inspection, which have emerged in other domains and that go
further than just using t-SNE in the embedding space, are not widely spread in
NLP.
  To reduce this gap, we investigate the application of DeepView, a method for
visualizing a part of the decision function together with a data set in two
dimensions, to the NLP domain. While in previous work, DeepView has been used
to inspect deep image classification models, we demonstrate how to apply it to
BERT-based NLP classifiers and investigate its usability in this domain,
including settings with adversarially perturbed input samples and pre-trained,
fine-tuned, and multi-task models.