The Blame Problem in Evaluating Local Explanations, and How to Tackle it
Amir Hossein Akhavan Rahnama
The number of local model-agnostic explanation techniques proposed has grown
rapidly recently. One main reason is that the bar for developing new
explainability techniques is low due to the lack of optimal evaluation
measures. Without rigorous measures, it is hard to have concrete evidence of
whether the new explanation techniques can significantly outperform their
predecessors. Our study proposes a new taxonomy for evaluating local
explanations: robustness, evaluation using ground truth from synthetic datasets
and interpretable models, model randomization, and human-grounded evaluation.
Using this proposed taxonomy, we highlight that all categories of evaluation
methods, except those based on the ground truth from interpretable models,
suffer from a problem we call the "blame problem." In our study, we argue that
this category of evaluation measure is a more reasonable method for evaluating
local model-agnostic explanations. However, we show that even this category of
evaluation measures has further limitations. The evaluation of local
explanations remains an open research problem.