SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approach
Mohammad N. S. Jahromi Satya. M. Muddamsetty Asta Sofie Stage Jarlner Anna Murphy Høgenhaug Thomas Gammeltoft-Hansen Thomas B. Moeslund
Explainable AI (XAI) aids in deciphering 'black-box' models. While several
methods have been proposed and evaluated primarily in the image domain, the
exploration of explainability in the text domain remains a growing research
area. In this paper, we delve into the applicability of XAI methods for the
text domain. In this context, the 'Similarity Difference and Uniqueness' (SIDU)
XAI method, recognized for its superior capability in localizing entire salient
regions in image-based classification is extended to textual data. The extended
method, SIDU-TXT, utilizes feature activation maps from 'black-box' models to
generate heatmaps at a granular, word-based level, thereby providing
explanations that highlight contextually significant textual elements crucial
for model predictions. Given the absence of a unified standard for assessing
XAI methods, this study applies a holistic three-tiered comprehensive
evaluation framework: Functionally-Grounded, Human-Grounded and
Application-Grounded, to assess the effectiveness of the proposed SIDU-TXT
across various experiments. We find that, in sentiment analysis task of a movie
review dataset, SIDU-TXT excels in both functionally and human-grounded
evaluations, demonstrating superior performance through quantitative and
qualitative analyses compared to benchmarks like Grad-CAM and LIME. In the
application-grounded evaluation within the sensitive and complex legal domain
of asylum decision-making, SIDU-TXT and Grad-CAM demonstrate comparable
performances, each with its own set of strengths and weaknesses. However, both
methods fall short of entirely fulfilling the sophisticated criteria of expert
expectations, highlighting the imperative need for additional research in XAI
methods suitable for such domains.