Counterfactual Explanations of Black-box Machine Learning Models using
  Causal Discovery with Applications to Credit Rating
Daisuke Takahashi Shohei Shimizu Takuma Tanaka
Explainable artificial intelligence (XAI) has helped elucidate the internal
mechanisms of machine learning algorithms, bolstering their reliability by
demonstrating the basis of their predictions. Several XAI models consider
causal relationships to explain models by examining the input-output
relationships of prediction models and the dependencies between features. The
majority of these models have been based their explanations on counterfactual
probabilities, assuming that the causal graph is known. However, this
assumption complicates the application of such models to real data, given that
the causal relationships between features are unknown in most cases. Thus, this
study proposed a novel XAI framework that relaxed the constraint that the
causal graph is known. This framework leveraged counterfactual probabilities
and additional prior information on causal structure, facilitating the
integration of a causal graph estimated through causal discovery methods and a
black-box classification model. Furthermore, explanatory scores were estimated
based on counterfactual probabilities. Numerical experiments conducted
employing artificial data confirmed the possibility of estimating the
explanatory score more accurately than in the absence of a causal graph.
Finally, as an application to real data, we constructed a classification model
of credit ratings assigned by Shiga Bank, Shiga prefecture, Japan. We
demonstrated the effectiveness of the proposed method in cases where the causal
graph is unknown.