Evaluating the overall sensitivity of saliency-based explanation methods
Harshinee Sriram Cristina Conati
We address the need to generate faithful explanations of "black box" Deep
Learning models. Several tests have been proposed to determine aspects of
faithfulness of explanation methods, but they lack cross-domain applicability
and a rigorous methodology. Hence, we select an existing test that is model
agnostic and is well-suited for comparing one aspect of faithfulness (i.e.,
sensitivity) of multiple explanation methods, and extend it by specifying
formal thresh-olds and building criteria to determine the over-all sensitivity
of the explanation method. We present examples of how multiple explanation
methods for Convolutional Neural Networks can be compared using this extended
methodology. Finally, we discuss the relationship between sensitivity and
faithfulness and consider how the test can be adapted to assess different
explanation methods in other domains.