A Comprehensive and Reliable Feature Attribution Method: Double-sided
  Remove and Reconstruct (DoRaR)
Dong Qin George Amariucai Daji Qiao Yong Guan Shen Fu
The limited transparency of the inner decision-making mechanism in deep
neural networks (DNN) and other machine learning (ML) models has hindered their
application in several domains. In order to tackle this issue, feature
attribution methods have been developed to identify the crucial features that
heavily influence decisions made by these black box models. However, many
feature attribution methods have inherent downsides. For example, one category
of feature attribution methods suffers from the artifacts problem, which feeds
out-of-distribution masked inputs directly through the classifier that was
originally trained on natural data points. Another category of feature
attribution method finds explanations by using jointly trained feature
selectors and predictors. While avoiding the artifacts problem, this new
category suffers from the Encoding Prediction in the Explanation (EPITE)
problem, in which the predictor's decisions rely not on the features, but on
the masks that selects those features. As a result, the credibility of
attribution results is undermined by these downsides. In this research, we
introduce the Double-sided Remove and Reconstruct (DoRaR) feature attribution
method based on several improvement methods that addresses these issues. By
conducting thorough testing on MNIST, CIFAR10 and our own synthetic dataset, we
demonstrate that the DoRaR feature attribution method can effectively bypass
the above issues and can aid in training a feature selector that outperforms
other state-of-the-art feature attribution methods. Our code is available at
https://github.com/dxq21/DoRaR.