Good-looking but Lacking Faithfulness: Understanding Local Explanation
  Methods through Trend-based Testing
Jinwen He Kai Chen Guozhu Meng Jiangshan Zhang Congyi Li
While enjoying the great achievements brought by deep learning (DL), people
are also worried about the decision made by DL models, since the high degree of
non-linearity of DL models makes the decision extremely difficult to
understand. Consequently, attacks such as adversarial attacks are easy to carry
out, but difficult to detect and explain, which has led to a boom in the
research on local explanation methods for explaining model decisions. In this
paper, we evaluate the faithfulness of explanation methods and find that
traditional tests on faithfulness encounter the random dominance problem, \ie,
the random selection performs the best, especially for complex data. To further
solve this problem, we propose three trend-based faithfulness tests and
empirically demonstrate that the new trend tests can better assess faithfulness
than traditional tests on image, natural language and security tasks. We
implement the assessment system and evaluate ten popular explanation methods.
Benefiting from the trend tests, we successfully assess the explanation methods
on complex data for the first time, bringing unprecedented discoveries and
inspiring future research. Downstream tasks also greatly benefit from the
tests. For example, model debugging equipped with faithful explanation methods
performs much better for detecting and correcting accuracy and security
problems.