We propose a method which can visually explain the classification decision of deep neural networks (DNNs). There are many proposed methods in machine learning and computer vision seeking to clarify the decision of machine learning black boxes, specifically DNNs.   All of these methods try to gain insight into why the network "chose class A" as an answer. Humans, when searching for explanations, ask two types of questions. The first question is, "Why did you choose this answer? " The second question asks, "Why did you not choose answer B over A?" The previously proposed methods are either not able to provide the latter directly or efficiently.

 We introduce a method capable of answering the second question both directly and efficiently. In this work, we limit the inputs to be images. In general, the proposed method generates explanations in the input space of any model capable of efficient evaluation and gradient evaluation. We provide results, showing the superiority of this approach for gaining insight into the inner representation of machine learning models. Deep neural networks (DNN) have shown extraordinary performance on computer vision tasks such as image classification BID25 BID23 BID24 BID7 , image segmentation BID3 , and image denoising BID29 . The first example of such a performance was on image classification, where it outperformed other computer vision methods which were carefully handcrafted for image classification BID12 . Following this success, DNNs continued to grow in popularity. Although the performances of DNNs on different tasks are getting close to human expertise BID17 and in some cases surpass them BID24 , there is hesitation to use them when interpretability of the results is important. Accuracy is a well-defined criterion but does not provide useful understandings of the actual inner workings of the network. If the deployment of a network may result in inputs whose distribution differs from that of the training or testing data, interpretability or explanations of the network's decisions can be important for securing human trust in the network.Explanations are important in settings such as medical treatments, system verification, and human training and teaching. Naturally, one way of getting an explanation is asking the direct question, "Why did the DNN choose this answer?" Humans often also seek contrasting explanations. For instance, they maybe more familiar with the contrasting answer, or they want to find the subtle differences in input which change the given answer to the contrasting one. This way of questioning can be phrased as, "Why did the DNN not choose B (over A)?" In this work, we present a framework to answer this type of question. We learn a model over the input space which is capable of generating synthetic samples similar to the input. Then, we ask how we can alter this synthetic input to change the classification outcome. Our proposed framework is not based on heuristics, does not need to change the given network, is applicable as long as the given model can handle backpropagation (no further requirements for layers), and can run much faster than methods with input perturbation. The only overhead of this method is the assumed availability of a latent model over the input. If this latent model is not available, we can learn such a model using generative adversarial methods or variational auto encoders. Learning this latent space needs to be done only a single time and is independent of the learned classifier to be explained. Our constrastive explanation method (CDeepEx) provides an effective method for querying a learned network to discover its learned representations and biases. We demonstrated the quality of our method, compared to other current methods and illustrated how these contrastive explanations can shed light on the robustness of a learned network.Asking a network contrastive questions of the form, "Why is this example not of class B?" can yield important information about how the network is making its decisions. Most previous explanation methods do not address this problem directly, and modifying their output to attempt to answer constrastive questions fails.Our formulation draws on three ideas: The explanation should be in the space of natural inputs, should be an example that is maximally ambiguous between the true and probe classes, and should not be confused with other classes. The method does not require knowledge of or heuristics related to the architecture or modality of the network. The explanations can point to unintended correlations in the input data that are expressed in the resulting learned network.
A method to answer "why not class B?" for explaining deep networks The paper proposes an approach to provide contrastive visual explanations for deep neural networks.