XInsight: Revealing Model Insights for GNNs with Flow-based Explanations
Eli Laird Ayesh Madushanka Elfi Kraka Corey Clark
Progress in graph neural networks has grown rapidly in recent years, with
many new developments in drug discovery, medical diagnosis, and recommender
systems. While this progress is significant, many networks are `black boxes'
with little understanding of the `what' exactly the network is learning. Many
high-stakes applications, such as drug discovery, require human-intelligible
explanations from the models so that users can recognize errors and discover
new knowledge. Therefore, the development of explainable AI algorithms is
essential for us to reap the benefits of AI.
  We propose an explainability algorithm for GNNs called eXplainable Insight
(XInsight) that generates a distribution of model explanations using GFlowNets.
Since GFlowNets generate objects with probabilities proportional to a reward,
XInsight can generate a diverse set of explanations, compared to previous
methods that only learn the maximum reward sample. We demonstrate XInsight by
generating explanations for GNNs trained on two graph classification tasks:
classifying mutagenic compounds with the MUTAG dataset and classifying acyclic
graphs with a synthetic dataset that we have open-sourced. We show the utility
of XInsight's explanations by analyzing the generated compounds using QSAR
modeling, and we find that XInsight generates compounds that cluster by
lipophilicity, a known correlate of mutagenicity. Our results show that
XInsight generates a distribution of explanations that uncovers the underlying
relationships demonstrated by the model. They also highlight the importance of
generating a diverse set of explanations, as it enables us to discover hidden
relationships in the model and provides valuable guidance for further analysis.