Knowledge graphs for empirical concept retrieval
Lenka Tětková Teresa Karen Scheidt Maria Mandrup Fogh Ellen Marie Gaunby Jørgensen Finn Årup Nielsen Lars Kai Hansen
Concept-based explainable AI is promising as a tool to improve the
understanding of complex models at the premises of a given user, viz.\ as a
tool for personalized explainability. An important class of concept-based
explainability methods is constructed with empirically defined concepts,
indirectly defined through a set of positive and negative examples, as in the
TCAV approach (Kim et al., 2018). While it is appealing to the user to avoid
formal definitions of concepts and their operationalization, it can be
challenging to establish relevant concept datasets. Here, we address this
challenge using general knowledge graphs (such as, e.g., Wikidata or WordNet)
for comprehensive concept definition and present a workflow for user-driven
data collection in both text and image domains. The concepts derived from
knowledge graphs are defined interactively, providing an opportunity for
personalization and ensuring that the concepts reflect the user's intentions.
We test the retrieved concept datasets on two concept-based explainability
methods, namely concept activation vectors (CAVs) and concept activation
regions (CARs) (Crabbe and van der Schaar, 2022). We show that CAVs and CARs
based on these empirical concept datasets provide robust and accurate
explanations. Importantly, we also find good alignment between the models'
representations of concepts and the structure of knowledge graphs, i.e., human
representations. This supports our conclusion that knowledge graph-based
concepts are relevant for XAI.