Occlusion Sensitivity Analysis with Augmentation Subspace Perturbation
  in Deep Feature Space
Pedro Valois Koichiro Niinuma Kazuhiro Fukui
Deep Learning of neural networks has gained prominence in multiple
life-critical applications like medical diagnoses and autonomous vehicle
accident investigations. However, concerns about model transparency and biases
persist. Explainable methods are viewed as the solution to address these
challenges. In this study, we introduce the Occlusion Sensitivity Analysis with
Deep Feature Augmentation Subspace (OSA-DAS), a novel perturbation-based
interpretability approach for computer vision. While traditional perturbation
methods make only use of occlusions to explain the model predictions, OSA-DAS
extends standard occlusion sensitivity analysis by enabling the integration
with diverse image augmentations. Distinctly, our method utilizes the output
vector of a DNN to build low-dimensional subspaces within the deep feature
vector space, offering a more precise explanation of the model prediction. The
structural similarity between these subspaces encompasses the influence of
diverse augmentations and occlusions. We test extensively on the ImageNet-1k,
and our class- and model-agnostic approach outperforms commonly used
interpreters, setting it apart in the realm of explainable AI.