How Case Based Reasoning Explained Neural Networks: An XAI Survey of Post-Hoc Explanation-by-Example in ANN-CBR Twins
---
paper_title: European Union regulations on algorithmic decision-making and a"right to explanation"

paper_content:

We summarize the potential impact that the European Union’s new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which “significantly affect” users. The law will also effectively create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for computer scientists to take the lead in designing algorithms and evaluation frameworks which avoid discrimination and enable explanation.
---
paper_title: Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

paper_content:

At the dawn of the fourth industrial revolution, we are witnessing a fast and widespread adoption of artificial intelligence (AI) in our daily life, which contributes to accelerating the shift towards a more algorithmic society. However, even with such unprecedented advancements, a key impediment to the use of AI-based systems is that they often lack transparency. Indeed, the black-box nature of these systems allows powerful predictions, but it cannot be directly explained. This issue has triggered a new debate on explainable AI (XAI). A research field holds substantial promise for improving trust and transparency of AI-based systems. It is recognized as the sine qua non for AI to continue making steady progress without disruption. This survey provides an entry point for interested researchers and practitioners to learn key aspects of the young and rapidly growing body of research related to XAI. Through the lens of the literature, we review the existing approaches regarding the topic, discuss trends surrounding its sphere, and present major research trajectories.
---
paper_title: Explanation and understanding.

paper_content:

The study of explanation, while related to intuitive theories, concepts, and mental models, offers important new perspectives on high-level thought. Explanations sort themselves into several distinct types corresponding to patterns of causation, content domains, and explanatory stances, all of which have cognitive consequences. Although explanations are necessarily incomplete—often dramatically so in laypeople—those gaps are difficult to discern. Despite such gaps and the failure to recognize them fully, people do have skeletal explanatory senses, often implicit, of the causal structure of the world. They further leverage those skeletal understandings by knowing how to access additional explanatory knowledge in other minds and by being particularly adept at using situational support to build explanations on the fly in real time. Across development and cultures, there are differences in preferred explanatory schemes, but rarely are any kinds of schemes completely unavailable to a group.
---
paper_title: Explanation in Artificial Intelligence: Insights from the Social Sciences

paper_content:

Abstract There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to provide more transparency to their algorithms. Much of this research is focused on explicitly explaining decisions or actions to a human observer, and it should not be controversial to say that looking at how humans explain to each other can serve as a useful starting point for explanation in artificial intelligence. However, it is fair to say that most work in explainable artificial intelligence uses only the researchers' intuition of what constitutes a ‘good’ explanation. There exist vast and valuable bodies of research in philosophy, psychology, and cognitive science of how people define, generate, select, evaluate, and present explanations, which argues that people employ certain cognitive biases and social expectations to the explanation process. This paper argues that the field of explainable artificial intelligence can build on this existing research, and reviews relevant papers from philosophy, cognitive psychology/science, and social psychology, which study these topics. It draws out some important findings, and discusses ways that these can be infused with work on explainable artificial intelligence.
---
paper_title: A Survey of Explanations in Recommender Systems

paper_content:

This paper provides a comprehensive review of explanations in recommender systems. We highlight seven possible advantages of an explanation facility, and describe how existing measures can be used to evaluate the quality of explanations. Since explanations are not independent of the recommendation process, we consider how the ways recommendations are presented may affect explanations. Next, we look at different ways of interacting with explanations. The paper is illustrated with examples of explanations throughout, where possible from existing applications.
---
paper_title: Towards A Rigorous Science of Interpretable Machine Learning

paper_content:

As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.
---
paper_title: Explanation in Case-Based Reasoning–Perspectives and Goals

paper_content:

We present an overview of different theories of explanation from the philosophy and cognitive science communities. Based on these theories, as well as models of explanation from the knowledge-based systems area, we present a framework for explanation in case-based reasoning (CBR) based on explanation goals. We propose ways that the goals of the user and system designer should be taken into account when deciding what is a good explanation for a given CBR system. Some general types of goals relevant to many CBR systems are identified, and used to survey existing methods of explanation in CBR. Finally, we identify some future challenges.
---
paper_title: A Survey of Methods for Explaining Black Box Models

paper_content:

In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.
---
paper_title: Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda

paper_content:

Advances in artificial intelligence, sensors and big data management have far-reaching societal impacts. As these systems augment our everyday lives, it becomes increasing-ly important for people to understand them and remain in control. We investigate how HCI researchers can help to develop accountable systems by performing a literature analysis of 289 core papers on explanations and explaina-ble systems, as well as 12,412 citing papers. Using topic modeling, co-occurrence and network analysis, we mapped the research space from diverse domains, such as algorith-mic accountability, interpretable machine learning, context-awareness, cognitive psychology, and software learnability. We reveal fading and burgeoning trends in explainable systems, and identify domains that are closely connected or mostly isolated. The time is ripe for the HCI community to ensure that the powerful new autonomous systems have intelligible interfaces built-in. From our results, we propose several implications and directions for future research to-wards this goal.
---
paper_title: Case-based explanation of non-case-based learning methods.

paper_content:

Abstract ::: We show how to generate case-based explanations for non-case-based learning methods such as artificial neural nets or decision trees. The method uses the trained model (e.g., the neural net or the decision tree) as a distance metric to determine which cases in the training set are most similar to the case that needs to be explained. This approach is well suited to medical domains, where it is important to understand predictions made by complex machine learning models, and where training and clinical practice makes users adept at case interpretation.
---
paper_title: Towards A Rigorous Science of Interpretable Machine Learning

paper_content:

As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.
---
paper_title: Meaningful Explanations of Black Box AI Decision Systems

paper_content:

Black box AI systems for automated decision making, often based on machine learning over (big) data, map a user’s features into a class or a score without exposing the reasons why. This is problematic not only for lack of transparency, but also for possible biases inherited by the algorithms from human prejudices and collection artifacts hidden in the training data, which may lead to unfair or wrong decisions. We focus on the urgent open challenge of how to construct meaningful explanations of opaque AI/ML systems, introducing the local-toglobal framework for black box explanation, articulated along three lines: (i) the language for expressing explanations in terms of logic rules, with statistical and causal interpretation; (ii) the inference of local explanations for revealing the decision rationale for a specific case, by auditing the black box in the vicinity of the target instance; (iii), the bottom-up generalization of many local explanations into simple global ones, with algorithms that optimize for quality and comprehensibility. We argue that the local-first approach opens the door to a wide variety of alternative solutions along different dimensions: a variety of data sources (relational, text, images, etc.), a variety of learning problems (multi-label classification, regression, scoring, ranking), a variety of languages for expressing meaningful explanations, a variety of means to audit a black box.
---
paper_title: The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification

paper_content:

We present the Bayesian Case Model (BCM), a general framework for Bayesian case-based reasoning (CBR) and prototype classification and clustering. BCM brings the intuitive power of CBR to a Bayesian generative framework. The BCM learns prototypes, the "quintessential" observations that best represent clusters in a dataset, by performing joint inference on cluster labels, prototypes and important features. Simultaneously, BCM pursues sparsity by learning subspaces, the sets of features that play important roles in the characterization of the prototypes. The prototype and subspace representation provides quantitative benefits in interpretability while preserving classification accuracy. Human subject experiments verify statistically significant improvements to participants' understanding when using explanations produced by BCM, compared to those given by prior art.
---
paper_title: Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

paper_content:

At the dawn of the fourth industrial revolution, we are witnessing a fast and widespread adoption of artificial intelligence (AI) in our daily life, which contributes to accelerating the shift towards a more algorithmic society. However, even with such unprecedented advancements, a key impediment to the use of AI-based systems is that they often lack transparency. Indeed, the black-box nature of these systems allows powerful predictions, but it cannot be directly explained. This issue has triggered a new debate on explainable AI (XAI). A research field holds substantial promise for improving trust and transparency of AI-based systems. It is recognized as the sine qua non for AI to continue making steady progress without disruption. This survey provides an entry point for interested researchers and practitioners to learn key aspects of the young and rapidly growing body of research related to XAI. Through the lens of the literature, we review the existing approaches regarding the topic, discuss trends surrounding its sphere, and present major research trajectories.
---
paper_title: Hybrid Neural Network and Expert Systems

paper_content:

Preface. Part I: Fundamentals of Hybrid Systems. 1. Overview of Neural and Symbolic Systems. 2. Research in Hybrid Neural and Symbolic Systems. 3. Models for Integrating Systems. Part II: Case Studies of Hybrid Neural Network and Expert Systems. 4. LAM Hybrid System for Window Glazing Design. 5. Hybrid Systems Approach to Nuclear Plant Monitoring. 6. Chemical Tank Control System. 7. Image Interpretation via Fusion of Heterogeneous Sources using a Hybrid Expert-Neural Network System. 8. Hybrid Systems for Multiple Target Recognition. Part III: Analysis and Guidelines. 9. Guidelines for Developing Hybrid Systems. 10. Tools and Development Systems. 11. Summary and the Future of Hybrid Neural Network and Expert Systems. References. Index.
---
paper_title: Data Mining: Practical Machine Learning Tools and Techniques

paper_content:

Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization
---
paper_title: Analogical Asides on Case-Based Reasoning

paper_content:

This paper explores some of the similarities and differences between cognitive models of analogy and case-based reasoning systems. I first point out a paradox in the treatment of adaptation in analogy and in case-based reasoning; a paradox which can be only resolved by expanding the role of adaptation in cognitive models of analogy. Some psychological research on the process of adaptation in human subjects is reported and then the implications of this research are propagated into analogy and then on into CBR. The argument is that some of the existing stages in CBR should be integrated into a more stream-lined architecture that would be more efficient than current schemes.
---
paper_title: Deep Residual Learning for Image Recognition

paper_content:

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
---
paper_title: A Survey of Explanations in Recommender Systems

paper_content:

This paper provides a comprehensive review of explanations in recommender systems. We highlight seven possible advantages of an explanation facility, and describe how existing measures can be used to evaluate the quality of explanations. Since explanations are not independent of the recommendation process, we consider how the ways recommendations are presented may affect explanations. Next, we look at different ways of interacting with explanations. The paper is illustrated with examples of explanations throughout, where possible from existing applications.
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: A systematic review and taxonomy of explanations in decision support and recommender systems

paper_content:

With the recent advances in the field of artificial intelligence, an increasing number of decision-making tasks are delegated to software systems. A key requirement for the success and adoption of such systems is that users must trust system choices or even fully automated decisions. To achieve this, explanation facilities have been widely investigated as a means of establishing trust in these systems since the early years of expert systems. With today’s increasingly sophisticated machine learning algorithms, new challenges in the context of explanations, accountability, and trust towards such systems constantly arise. In this work, we systematically review the literature on explanations in advice-giving systems. This is a family of systems that includes recommender systems, which is one of the most successful classes of advice-giving software in practice. We investigate the purposes of explanations as well as how they are generated, presented to users, and evaluated. As a result, we derive a novel comprehensive taxonomy of aspects to be considered when designing explanation facilities for current and future decision support systems. The taxonomy includes a variety of different facets, such as explanation objective, responsiveness, content and presentation. Moreover, we identified several challenges that remain unaddressed so far, for example related to fine-grained issues associated with the presentation of explanations and how explanation facilities are evaluated.
---
paper_title: Explanation in Case-Based Reasoning–Perspectives and Goals

paper_content:

We present an overview of different theories of explanation from the philosophy and cognitive science communities. Based on these theories, as well as models of explanation from the knowledge-based systems area, we present a framework for explanation in case-based reasoning (CBR) based on explanation goals. We propose ways that the goals of the user and system designer should be taken into account when deciding what is a good explanation for a given CBR system. Some general types of goals relevant to many CBR systems are identified, and used to survey existing methods of explanation in CBR. Finally, we identify some future challenges.
---
paper_title: Visualizing and Understanding Convolutional Networks

paper_content:

Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark Krizhevsky et al. [18]. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we explore both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. Used in a diagnostic role, these visualizations allow us to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. We also perform an ablation study to discover the performance contribution from different model layers. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.
---
paper_title: A Survey of Methods for Explaining Black Box Models

paper_content:

In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.
---
paper_title: Intriguing properties of neural networks

paper_content:

Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While their expressiveness is the reason they succeed, it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. In this paper we report two such properties. ::: First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. ::: Second, we find that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input.
---
paper_title: Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

paper_content:

We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’ in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a ‘stronger’ deep network from a ‘weaker’ one even when both make identical predictions. Our code is available at https://github.com/ramprs/grad-cam/, along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265–290. Springer, 2015) (http://gradcam.cloudcv.org) and a video at http://youtu.be/COjUB9Izk6E.
---
paper_title: Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda

paper_content:

Advances in artificial intelligence, sensors and big data management have far-reaching societal impacts. As these systems augment our everyday lives, it becomes increasing-ly important for people to understand them and remain in control. We investigate how HCI researchers can help to develop accountable systems by performing a literature analysis of 289 core papers on explanations and explaina-ble systems, as well as 12,412 citing papers. Using topic modeling, co-occurrence and network analysis, we mapped the research space from diverse domains, such as algorith-mic accountability, interpretable machine learning, context-awareness, cognitive psychology, and software learnability. We reveal fading and burgeoning trends in explainable systems, and identify domains that are closely connected or mostly isolated. The time is ripe for the HCI community to ensure that the powerful new autonomous systems have intelligible interfaces built-in. From our results, we propose several implications and directions for future research to-wards this goal.
---
paper_title: Deep Learning for Case-based Reasoning through Prototypes: A Neural Network that Explains its Predictions

paper_content:

Deep neural networks are widely used for classification. These deep models often suffer from a lack of interpretability -- they are particularly difficult to understand because of their non-linear nature. As a result, neural networks are often treated as "black box" models, and in the past, have been trained purely to optimize the accuracy of predictions. In this work, we create a novel network architecture for deep learning that naturally explains its own reasoning for each prediction. This architecture contains an autoencoder and a special prototype layer, where each unit of that layer stores a weight vector that resembles an encoded training input. The encoder of the autoencoder allows us to do comparisons within the latent space, while the decoder allows us to visualize the learned prototypes. The training objective has four terms: an accuracy term, a term that encourages every prototype to be similar to at least one encoded input, a term that encourages every encoded input to be close to at least one prototype, and a term that encourages faithful reconstruction by the autoencoder. The distances computed in the prototype layer are used as part of the classification process. Since the prototypes are learned during training, the learned network naturally comes with explanations for each prediction, and the explanations are loyal to what the network actually computes.
---
paper_title: A local weighting method to the integration of neural network and case based reasoning

paper_content:

Our aim is to build an integrated learning framework of neural network and case based reasoning. The main idea is that feature weights for case based reasoning can be evaluated using neural networks. In our previous method, we derived the feature weight set from the trained neural network and the training data so that the feature weight is constant for all queries. In this paper, we propose a local feature weighting method using a neural network. The neural network guides the case based reasoning by providing case-specific weights to the learning process. We developed a learning process to get the local weights using the neural network and showed the performance of our learning system using the sinusoidal dataset.
---
paper_title: A connectionist fuzzy case-based reasoning model

paper_content:

This paper presents a new version of an existing hybrid model for the development of knowledge-based systems, where case-based reasoning is used as a problem solver. Numeric predictive attributes are modeled in terms of fuzzy sets to define neurons in an associative Artificial Neural Network (ANN). After the Fuzzy-ANN is trained, its weights and the membership degrees in the training examples are used to automatically generate a local distance function and an attribute weighting scheme. Using this distance function and following the Nearest Neighbor rule, a new hybrid Connectionist Fuzzy Case-Based Reasoning model is defined. Experimental results show that the model proposed allows to develop knowledge-based systems with a higher accuracy than when using the original model. The model takes the advantages of the approaches used, providing a more natural framework to include expert knowledge by using linguistic terms.
---
paper_title: Case-based explanation of non-case-based learning methods.

paper_content:

Abstract ::: We show how to generate case-based explanations for non-case-based learning methods such as artificial neural nets or decision trees. The method uses the trained model (e.g., the neural net or the decision tree) as a distance metric to determine which cases in the training set are most similar to the case that needs to be explained. This approach is well suited to medical domains, where it is important to understand predictions made by complex machine learning models, and where training and clinical practice makes users adept at case interpretation.
---
paper_title: A machine learning approach to yield management in semiconductor manufacturing

paper_content:

Yield improvement is one of the most important topics in semiconductor manufacturing. Traditional statistical methods are no longer feasible nor efficient, if possible, in analysing the vast amount...
---
paper_title: Towards integration of memory based learning and neural networks

paper_content:

We propose a hybrid prediction system of neural network (NN) and memory based learning (MBR). NN and MBR are frequently applied to data mining with various objectives. NN and MBR can be directly applied to classification and regression without additional transformation mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. In our hybrid system of NN and MBR, the feature weight set which is calculated from the trained NN plays the core role in connecting both learning strategies and the explanation on prediction can be given by obtaining and presenting the most similar examples from the case base. Experimental results show that the hybrid system has a high potential in solving data mining problems.
---
paper_title: Memory and neural network based expert system

paper_content:

Abstract We suggest a hybrid expert system of memory and neural network based learning. Neural network (NN) and memory based reasoning (MBR) have common advantages over other learning strategies. NN and MBR can be directly applied to the classification and regression problem without additional transform mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. Unfortunately, they have an achilles tendon. The knowledge representation of NN is unreadable to human being and this ‘black box’ property restricts the application of NN to areas which needs proper explanations as well as precise predictions. On the other hand, MBR suffers from the feature-weighting problem. When MBR measures the distance between cases, some features should be treated more importantly than others. Although previous researchers provide several feature-weighting mechanisms to overcome the difficulty, those methods were mainly applicable only to the classification problem. In our hybrid system of NN and MBR, the feature weight set calculated from the trained neural network plays the core role in connecting both the learning strategies. Moreover, the explanation on prediction can be given by presenting the most similar cases from the case base. In this paper, we present the basic idea of the hybrid system. We also present an application example with a wafer yield prediction system for semiconductor manufacturing. Experimental results show that the hybrid system predicts the yield with relatively high accuracy and is capable of learning adaptively to changing behavior of the manufacturing system.
---
paper_title: Combining a neural network with case-based reasoning in a diagnostic system.

paper_content:

Abstract This paper presents a new approach for integrating case-based reasoning (CBR) with a neural network (NN) in diagnostic systems. When solving a new problem, the neural network is used to make hypotheses and to guide the CBR module in the search for a similar previous case that supports one of the hypotheses. The knowledge acquired by the network is interpreted and mapped into symbolic diagnosis descriptors , which are kept and used by the system to determine whether a final answer is credible, and to build explanations for the reasoning carried out. The NN-CBR model has been used in the development of a system for the diagnosis of congenital heart diseases (CHD). The system has been evaluated using two cardiological databases with a total of 214 CHD cases. Three other well-known databases have been used to evaluate the NN-CBR approach further. The hybrid system manages to solve problems that cannot be solved by the neural network with a good level of accuracy. Additionally, the hybrid system suggests some solutions for common CBR problems, such as indexing and retrieval, as well as for neural network problems, such as the interpretation of the knowledge stored in a neural network and the explanation of reasoning.
---
paper_title: Case-based reasoning and neural network based expert system for personalization

paper_content:

Abstract We suggest a hybrid expert system of case-based reasoning (CBR) and neural network (NN) for symbolic domain. In previous research, we proposed a hybrid system of memory and neural network based learning. In the system, the feature weights are extracted from the trained neural network, and used to improve retrieval accuracy of case-based reasoning. However, this system has worked best in domains in which all features had numeric values. When the feature values are symbolic, nearest neighbor methods typically resort to much simpler metrics, such as counting the features that match. A more sophisticated treatment of the feature space is required in symbolic domains. We propose feature-weighted CBR with neural network, which uses value difference metric (VDM) as distance function for symbolic features. In our system, the feature weight set calculated from the trained neural network plays the core role in connecting both the learning strategies. Moreover, the explanation on prediction can be given by presenting the most similar cases from the case base. To validate our system, illustrative experimental results are presented. We use datasets from the UCI machine learning archive for experiments. Finally, we present an application with a personalized counseling system for cosmetic industry whose questionnaires have symbolic features. Feature-weighted CBR with neural network predicts the five elements, which show customers’ character and physical constitution, with relatively high accuracy and expert system for personalization recommends personalized make-up style, color, life style and products.
---
paper_title: Answering with Cases: A CBR Approach to Deep Learning

paper_content:

Every year tenths of thousands of customer support engineers around the world deal with, and proactively solve, complex help-desk tickets. Daily, almost every customer support expert will turn his/her attention to a prioritization strategy, to achieve the best possible result. To assist with this, in this paper we describe a novel case-based reasoning application to address the tasks of: high solution accuracy and shorter prediction resolution time. We describe how appropriate cases can be generated to assist engineers and how our solution can scale over time to produce domain-specific reusable cases for similar problems. Our work is evaluated using data from 5000 cases from the automotive industry.
---
paper_title: Explanation in Case-Based Reasoning–Perspectives and Goals

paper_content:

We present an overview of different theories of explanation from the philosophy and cognitive science communities. Based on these theories, as well as models of explanation from the knowledge-based systems area, we present a framework for explanation in case-based reasoning (CBR) based on explanation goals. We propose ways that the goals of the user and system designer should be taken into account when deciding what is a good explanation for a given CBR system. Some general types of goals relevant to many CBR systems are identified, and used to survey existing methods of explanation in CBR. Finally, we identify some future challenges.
---
paper_title: A hybrid approach of neural network and memory-based learning to data mining

paper_content:

We propose a hybrid prediction system of neural network and memory-based learning. Neural network (NN) and memory-based reasoning (MBR) are frequently applied to data mining with various objectives. They have common advantages over other learning strategies. NN and MBR can be directly applied to classification and regression without additional transformation mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. Unfortunately, they have shortcomings when applied to data mining tasks. Though the neural network is considered as one of the most powerful and universal predictors, the knowledge representation of NN is unreadable to humans, and this "black box" property restricts the application of NN to data mining problems, which require proper explanations for the prediction. On the other hand, MBR suffers from the feature-weighting problem. When MBR measures the distance between cases, some input features should be treated as more important than other features. Feature weighting should be executed prior to prediction in order to provide the information on the feature importance. In our hybrid system of NN and MBR, the feature weight set, which is calculated from the trained neural network, plays the core role in connecting both learning strategies, and the explanation for prediction can be given by obtaining and presenting the most similar examples from the case base. Moreover, the proposed system has advantages in the typical data mining problems such as scalability to large datasets, high dimensions, and adaptability to dynamic situations. Experimental results show that the hybrid system has a high potential in solving data mining problems.
---
paper_title: A Hybrid Case-based Model for Forecasting

paper_content:

An investigation is described into the application of artificial intelligence to forecasting in the domain of oceanography. A hybrid approach to forecasting the thermal structure of the water ahead of a moving vessel is presented which combines the ability of a case-based reasoning system for identifying previously encountered similar situations and the generalizing ability of an artificial neural network to guide the adaptation stage of the case-based reasoning mechanism. The system has been successfully tested in real time in the Atlantic Ocean; the results obtained are presented and compared with those derived from other forecasting methods.
---
paper_title: An Automated Hybrid CBR System for Forecasting

paper_content:

A hybrid neuro-symbolic problem solving model is presented in which the aim is to forecast parameters of a complex and dynamic environment in an unsupervised way. In situations in which the rules that determine a system are unknown, the prediction of the parameter values that determine the characteristic behaviour of the system can be a problematic task. The proposed system employs a case-based reasoning model that incorporates a growing cell structures network, a radial basis function network and a set of Sugeno fuzzy models to provide an accurate prediction. Each of these techniques is used in a different stage of the reasoning cycle of the case-based reasoning system to retrieve, to adapt and to review the proposed solution to the problem. This system has been used to predict the red tides that appear in the coastal waters of the north west of the Iberian Peninsula. The results obtained from those experiments are presented.
---
paper_title: Data mining using example-based methods in oceanographic forecast models

paper_content:

The abstract presents a hybrid system that has proved capable of creating a prediction detailing the physical interactions occurring in a rapidly changing oceanographic environment. The aim of the system is to identify and forecast the thermal structure of the water ahead of an ongoing vessel. The work focuses on the development of a system for forecasting the behaviour of complex environments, in which the underling knowledge of the domain is not completely available, the rules governing the system are fuzzy and the sets of data samples are limited and incomplete. The paper presents a hybrid approach that combines the ability of a case based reasoning system (CBR) for Selecting previous similar situations and the generalising ability of artificial neural networks (ANN) to guide the adaptation stage of the case base reasoning system. The system has been successfully tested in the Atlantic Ocean in September 1997.
---
paper_title: Modular Integration of Connectionist and Symbolic Processing in Knowledge-Based Systems

paper_content:

MIX is an ESPRIT project aimed at developing strategies and tools for integrating symbolic and neural methods in hybrid systems. The project arose from the observation that current hybrid systems are generally small-scale experimental systems which couple one symbolic and one connectionist model, often in an ad hoc fashion. Hence the objective of building a versatile testbed for the design, prototyping and assessment of a variety of hybrid models or architectures, in particular those which combine diverse neural network models with rule/model-based, cased-based, and fuzzy reasoning. A multiagent approach has been chosen to facilitate modular implementation of these hybrid models, which will be tested in the context of real-world applications in the steel and automobile industries.
---
paper_title: MBNR: Case-Based Reasoning with Local Feature Weighting by Neural Network

paper_content:

Our aim is to build an integrated learning framework of neural network and case-based reasoning. The main idea is that feature weights for case-based reasoning can be evaluated by neural networks. In this paper, we propose MBNR (Memory-Based Neural Reasoning), case-based reasoning with local feature weighting by neural network. In our method, the neural network guides the case-based reasoning by providing case-specific weights to the learning process. We developed a learning algorithm to train the neural network to learn the case-specific local weighting patterns for case-based reasoning. We showed the performance of our learning system using four datasets.
---
paper_title: Case-based explanation of non-case-based learning methods.

paper_content:

Abstract ::: We show how to generate case-based explanations for non-case-based learning methods such as artificial neural nets or decision trees. The method uses the trained model (e.g., the neural net or the decision tree) as a distance metric to determine which cases in the training set are most similar to the case that needs to be explained. This approach is well suited to medical domains, where it is important to understand predictions made by complex machine learning models, and where training and clinical practice makes users adept at case interpretation.
---
paper_title: Memory and neural network based expert system

paper_content:

Abstract We suggest a hybrid expert system of memory and neural network based learning. Neural network (NN) and memory based reasoning (MBR) have common advantages over other learning strategies. NN and MBR can be directly applied to the classification and regression problem without additional transform mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. Unfortunately, they have an achilles tendon. The knowledge representation of NN is unreadable to human being and this ‘black box’ property restricts the application of NN to areas which needs proper explanations as well as precise predictions. On the other hand, MBR suffers from the feature-weighting problem. When MBR measures the distance between cases, some features should be treated more importantly than others. Although previous researchers provide several feature-weighting mechanisms to overcome the difficulty, those methods were mainly applicable only to the classification problem. In our hybrid system of NN and MBR, the feature weight set calculated from the trained neural network plays the core role in connecting both the learning strategies. Moreover, the explanation on prediction can be given by presenting the most similar cases from the case base. In this paper, we present the basic idea of the hybrid system. We also present an application example with a wafer yield prediction system for semiconductor manufacturing. Experimental results show that the hybrid system predicts the yield with relatively high accuracy and is capable of learning adaptively to changing behavior of the manufacturing system.
---
paper_title: A local weighting method to the integration of neural network and case based reasoning

paper_content:

Our aim is to build an integrated learning framework of neural network and case based reasoning. The main idea is that feature weights for case based reasoning can be evaluated using neural networks. In our previous method, we derived the feature weight set from the trained neural network and the training data so that the feature weight is constant for all queries. In this paper, we propose a local feature weighting method using a neural network. The neural network guides the case based reasoning by providing case-specific weights to the learning process. We developed a learning process to get the local weights using the neural network and showed the performance of our learning system using the sinusoidal dataset.
---
paper_title: A hybrid CBR classification model by integrating ANN into CBR

paper_content:

Case-based reasoning (CBR) is an artificial intelligent approach to problem solving and learning, which understands and extracts knowledge from past cases. However, CBR faces the challenge of assigning weights to the features to measure similarity between cases effectively and correctly. Integration of inherent learning capability of artificial neural networks (ANNs) to help the CBR in attributing the correct and appropriate weights to the features is likely to improve the performance of the standard CBR approach. This paper integrates back propagation neural network (BPNN) into CBR in an innovative way to develop an efficient model for classification tasks. The implementation of integration of NN and CBR for classification tasks is done by building training and testing datasets and optimising NN architecture in terms of number of neurons in hidden layer. This paper investigates the integration of multi-layer BP neural network and CBR. The experimental results obtained with the proposed hybrid model are compared with that of standard CBR, CBR with value difference matrix (VDM) and one existing CBR with BPNN approaches. The superiority of the proposed hybrid CBR model is established to others. The performance of the proposed model is validated with four datasets.
---
paper_title: A machine learning approach to yield management in semiconductor manufacturing

paper_content:

Yield improvement is one of the most important topics in semiconductor manufacturing. Traditional statistical methods are no longer feasible nor efficient, if possible, in analysing the vast amount...
---
paper_title: Towards integration of memory based learning and neural networks

paper_content:

We propose a hybrid prediction system of neural network (NN) and memory based learning (MBR). NN and MBR are frequently applied to data mining with various objectives. NN and MBR can be directly applied to classification and regression without additional transformation mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. In our hybrid system of NN and MBR, the feature weight set which is calculated from the trained NN plays the core role in connecting both learning strategies and the explanation on prediction can be given by obtaining and presenting the most similar examples from the case base. Experimental results show that the hybrid system has a high potential in solving data mining problems.
---
paper_title: Memory and neural network based expert system

paper_content:

Abstract We suggest a hybrid expert system of memory and neural network based learning. Neural network (NN) and memory based reasoning (MBR) have common advantages over other learning strategies. NN and MBR can be directly applied to the classification and regression problem without additional transform mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. Unfortunately, they have an achilles tendon. The knowledge representation of NN is unreadable to human being and this ‘black box’ property restricts the application of NN to areas which needs proper explanations as well as precise predictions. On the other hand, MBR suffers from the feature-weighting problem. When MBR measures the distance between cases, some features should be treated more importantly than others. Although previous researchers provide several feature-weighting mechanisms to overcome the difficulty, those methods were mainly applicable only to the classification problem. In our hybrid system of NN and MBR, the feature weight set calculated from the trained neural network plays the core role in connecting both the learning strategies. Moreover, the explanation on prediction can be given by presenting the most similar cases from the case base. In this paper, we present the basic idea of the hybrid system. We also present an application example with a wafer yield prediction system for semiconductor manufacturing. Experimental results show that the hybrid system predicts the yield with relatively high accuracy and is capable of learning adaptively to changing behavior of the manufacturing system.
---
paper_title: CBR for Modeling Complex Systems

paper_content:

This paper describes how CBR can be used to compare, reuse, and adapt inductive models that represent complex systems. Complex systems are not well understood and therefore require models for their manipulation and understanding. We propose an approach to address the challenges for using CBR in this context, which relate to finding similar inductive models (solutions) to represent similar complex systems (problems). The purpose is to improve the modeling task by considering the quality of different models to represent a system based on the similarity to a system that was successfully modeled. The revised and confirmed suitability of a model can become additional evidence of similarity between two complex systems, resulting in an increased understanding of a domain. This use of CBR supports tasks (e.g., diagnosis, prediction) that inductive or mathematical models alone cannot perform. We validate our approach by modeling software systems, and illustrate its potential significance for biological systems.
---
paper_title: Case-based reasoning and neural network based expert system for personalization

paper_content:

Abstract We suggest a hybrid expert system of case-based reasoning (CBR) and neural network (NN) for symbolic domain. In previous research, we proposed a hybrid system of memory and neural network based learning. In the system, the feature weights are extracted from the trained neural network, and used to improve retrieval accuracy of case-based reasoning. However, this system has worked best in domains in which all features had numeric values. When the feature values are symbolic, nearest neighbor methods typically resort to much simpler metrics, such as counting the features that match. A more sophisticated treatment of the feature space is required in symbolic domains. We propose feature-weighted CBR with neural network, which uses value difference metric (VDM) as distance function for symbolic features. In our system, the feature weight set calculated from the trained neural network plays the core role in connecting both the learning strategies. Moreover, the explanation on prediction can be given by presenting the most similar cases from the case base. To validate our system, illustrative experimental results are presented. We use datasets from the UCI machine learning archive for experiments. Finally, we present an application with a personalized counseling system for cosmetic industry whose questionnaires have symbolic features. Feature-weighted CBR with neural network predicts the five elements, which show customers’ character and physical constitution, with relatively high accuracy and expert system for personalization recommends personalized make-up style, color, life style and products.
---
paper_title: Towards A Rigorous Science of Interpretable Machine Learning

paper_content:

As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.
---
paper_title: A hybrid approach of neural network and memory-based learning to data mining

paper_content:

We propose a hybrid prediction system of neural network and memory-based learning. Neural network (NN) and memory-based reasoning (MBR) are frequently applied to data mining with various objectives. They have common advantages over other learning strategies. NN and MBR can be directly applied to classification and regression without additional transformation mechanisms. They also have strength in learning the dynamic behavior of the system over a period of time. Unfortunately, they have shortcomings when applied to data mining tasks. Though the neural network is considered as one of the most powerful and universal predictors, the knowledge representation of NN is unreadable to humans, and this "black box" property restricts the application of NN to data mining problems, which require proper explanations for the prediction. On the other hand, MBR suffers from the feature-weighting problem. When MBR measures the distance between cases, some input features should be treated as more important than other features. Feature weighting should be executed prior to prediction in order to provide the information on the feature importance. In our hybrid system of NN and MBR, the feature weight set, which is calculated from the trained neural network, plays the core role in connecting both learning strategies, and the explanation for prediction can be given by obtaining and presenting the most similar examples from the case base. Moreover, the proposed system has advantages in the typical data mining problems such as scalability to large datasets, high dimensions, and adaptability to dynamic situations. Experimental results show that the hybrid system has a high potential in solving data mining problems.
---
paper_title: A Survey of Methods for Explaining Black Box Models

paper_content:

In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.
---
paper_title: Hybrid case-based reasoning system by cost-sensitive neural network for classification

paper_content:

Case-based reasoning (CBR) is an artificial intelligent approach to learning and problem-solving, which solves a target problem by relating past similar solved problems. But it faces the challenge of weights assignment to features to measure similarity between cases. There are many methods to overcome this feature weighting problem of CBR. However, neural network’s pruning is one of the powerful and useful methods to overcome this feature weighting problem, which extracts feature weights from trained neural network without losing the generality of training set by four popular mechanisms: sensitivity, activity, saliency and relevance. It is habitually assumed that the training sets used for learning are balanced. However, this hypothesis is not always true in real-world applications, and hence, the tendency is to yield classification models that are biased toward the overrepresented class. Therefore, a hybrid CBR system is proposed in this paper to overcome this problem, which adopts a cost-sensitive back-propagation neural network (BPNN) in network pruning to find feature weights. These weights are used in CBR. A single cost parameter is used by the cost-sensitive BPNN to distinguish the importance of class errors. A balanced decision boundary is generated by the cost parameter using prior information. Thus, the class imbalance problem of network pruning is overcome to improve the accuracy of the hybrid CBR. From the empirical results, it is observed that the performance of the proposed hybrid CBR system is better than the hybrid CBR by standard neural network. The performance of the proposed hybrid system is validated with seven datasets.
---
paper_title: A Case-based Reasoning with Feature Weights Derived by BP Network

paper_content:

Case-based reasoning (CBR) is a methodology for problem solving and decision-making in complex and changing environments. This study investigates the performance of a hybrid case-based reasoning method that integrates a multi-layer BP neural network with case-based reasoning (CBR) algorithms for derivatives feature weights. This approach is applied to fault detection and diagnosis (FDD) system involves the examination of several criteria. The correct identification of the underlying mechanism of a fault is an important step in the entire fault analysis process. The trained BP neural network provides the basis to obtain attribute weights, whereas CBR serves as a classifier to identify the fault mechanism. Different parameters of the hybrid methods were varied to study their effect. The results indicate that better performance could be achieved by the proposed hybrid method than that using conventional CBR alone.
---
paper_title: MBNR: Case-Based Reasoning with Local Feature Weighting by Neural Network

paper_content:

Our aim is to build an integrated learning framework of neural network and case-based reasoning. The main idea is that feature weights for case-based reasoning can be evaluated by neural networks. In this paper, we propose MBNR (Memory-Based Neural Reasoning), case-based reasoning with local feature weighting by neural network. In our method, the neural network guides the case-based reasoning by providing case-specific weights to the learning process. We developed a learning algorithm to train the neural network to learn the case-specific local weighting patterns for case-based reasoning. We showed the performance of our learning system using four datasets.
---
paper_title: Hybrid expert system using case based reasoning and neural network for classification

paper_content:

Abstract Case Based Reasoning (CBR) is an analogical reasoning method, which solves problems by relating some previously solved problems to a current unsolved problem to draw analogical inferences for problem solving. But CBR faces the challenge of assigning weights to the features to measure similarity between a current unsolved case and cases stored in the case base effectively and correctly. The concept of neural network’s pruning is already used to sort out feature weighting problem in CBR. But it loses generality and actual elicited knowledge in the ANN’s links. This work proposes a method to extract symbolic weights from a trained neural network by observing the whole trained neural network as an AND/OR graph and then finds solution for each node that becomes the weight of a corresponding node. The proposed feature weighting mechanism is used in CBR to build a hybrid expert system for classification task and the performance of the proposed hybrid system is compared with that with other feature weighting mechanisms. The performance is validated on swine flu dataset and ionosphere, sonar and heart datasets collected from UCI repository. From the empirical results it is observed that in all the experiments the proposed feature weighting mechanism outperforms most of the earlier weighting mechanisms extracted from trained neural network.
---
paper_title: Case-based explanation of non-case-based learning methods.

paper_content:

Abstract ::: We show how to generate case-based explanations for non-case-based learning methods such as artificial neural nets or decision trees. The method uses the trained model (e.g., the neural net or the decision tree) as a distance metric to determine which cases in the training set are most similar to the case that needs to be explained. This approach is well suited to medical domains, where it is important to understand predictions made by complex machine learning models, and where training and clinical practice makes users adept at case interpretation.
---
paper_title: An evaluation of machine-learning methods for predicting pneumonia mortality

paper_content:

Abstract This paper describes the application of eight statistical and machine-learning methods to derive computer models for predicting mortality of hospital patients with pneumonia from their findings at initial presentation. The eight models were each constructed based on 9847 patient cases and they were each evaluated on 4352 additional cases. The primary evaluation metric was the error in predicted survival as a function of the fraction of patients predicted to survive. This metric is useful in assessing a model's potential to assist a clinician in deciding whether to treat a given patient in the hospital or at home. We examined the error rates of the models when predicting that a given fraction of patients will survive. We examined survival fractions between 0.1 and 0.6. Over this range, each model's predictive error rate was within 1% of the error rate of every other model. When predicting that approximately 30% of the patients will survive, all the models have an error rate of less than 1.5%. The models are distinguished more by the number of variables and parameters that they contain than by their error rates; these differences suggest which models may be the most amenable to future implementation as paper-based guidelines.
---
paper_title: Explaining Explanations in AI

paper_content:

Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions. These models are a useful pedagogical device for teaching trained professionals how to predict what decisions will be made by the complex system, and most importantly how the system might break. However, when considering any such model it's important to remember Box's maxim that "All models are wrong but some are useful." We focus on the distinction between these models and explanations in philosophy and sociology. These models can be understood as a "do it yourself kit" for explanations, allowing a practitioner to directly answer "what if questions" or generate contrastive explanations without external assistance. Although a valuable ability, giving these models as explanations appears more difficult than necessary, and other forms of explanation may not have the same trade-offs. We contrast the different schools of thought on what makes an explanation, and suggest that machine learning might benefit from viewing the problem more broadly.
---
paper_title: Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission

paper_content:

In machine learning often a tradeoff must be made between accuracy and intelligibility. More accurate models such as boosted trees, random forests, and neural nets usually are not intelligible, but more intelligible models such as logistic regression, naive-Bayes, and single decision trees often have significantly worse accuracy. This tradeoff sometimes limits the accuracy of models that can be applied in mission-critical applications such as healthcare where being able to understand, validate, edit, and trust a learned model is important. We present two case studies where high-performance generalized additive models with pairwise interactions (GA2Ms) are applied to real healthcare problems yielding intelligible models with state-of-the-art accuracy. In the pneumonia risk prediction case study, the intelligible model uncovers surprising patterns in the data that previously had prevented complex learned models from being fielded in this domain, but because it is intelligible and modular allows these patterns to be recognized and removed. In the 30-day hospital readmission case study, we show that the same methods scale to large datasets containing hundreds of thousands of patients and thousands of attributes while remaining intelligible and providing accuracy comparable to the best (unintelligible) machine learning methods.
---
paper_title: Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning

paper_content:

Deep neural networks (DNNs) enable innovative applications of machine learning like image recognition, machine translation, or malware detection. However, deep learning is often criticized for its lack of robustness in adversarial settings (e.g., vulnerability to adversarial inputs) and general inability to rationalize its predictions. In this work, we exploit the structure of deep learning to enable new learning-based inference and decision strategies that achieve desirable properties such as robustness and interpretability. We take a first step in this direction and introduce the Deep k-Nearest Neighbors (DkNN). This hybrid classifier combines the k-nearest neighbors algorithm with representations of the data learned by each layer of the DNN: a test input is compared to its neighboring training points according to the distance that separates them in the representations. We show the labels of these neighboring points afford confidence estimates for inputs outside the model's training manifold, including on malicious inputs like adversarial examples--and therein provides protections against inputs that are outside the models understanding. This is because the nearest neighbors can be used to estimate the nonconformity of, i.e., the lack of support for, a prediction in the training data. The neighbors also constitute human-interpretable explanations of predictions. We evaluate the DkNN algorithm on several datasets, and show the confidence estimates accurately identify inputs outside the model, and that the explanations provided by nearest neighbors are intuitive and useful in understanding model failures.
---
paper_title: Case-based explanation of non-case-based learning methods.

paper_content:

Abstract ::: We show how to generate case-based explanations for non-case-based learning methods such as artificial neural nets or decision trees. The method uses the trained model (e.g., the neural net or the decision tree) as a distance metric to determine which cases in the training set are most similar to the case that needs to be explained. This approach is well suited to medical domains, where it is important to understand predictions made by complex machine learning models, and where training and clinical practice makes users adept at case interpretation.
---
paper_title: Explanation oriented retrieval

paper_content:

This paper is based on the observation that the nearest neighbour in a case-based prediction system may not be the best case to explain a prediction. This observation is based on the notion of a decision surface (i.e. class boundary) and the idea that cases located between the target case and the decision surface are more convincing as support for explanation. This motivates the idea of explanation utility, a metric that may be different to the similarity metric used for nearest neighbour retrieval. In this paper we present an explanation utility framework and present detailed examples of how it is used in two medical decision-support tasks. These examples show how this notion of explanation utility sometimes select cases other than the nearest neighbour for use in explanation and how these cases are more convincing as explanations.
---
paper_title: Explaining Explanations in AI

paper_content:

Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions. These models are a useful pedagogical device for teaching trained professionals how to predict what decisions will be made by the complex system, and most importantly how the system might break. However, when considering any such model it's important to remember Box's maxim that "All models are wrong but some are useful." We focus on the distinction between these models and explanations in philosophy and sociology. These models can be understood as a "do it yourself kit" for explanations, allowing a practitioner to directly answer "what if questions" or generate contrastive explanations without external assistance. Although a valuable ability, giving these models as explanations appears more difficult than necessary, and other forms of explanation may not have the same trade-offs. We contrast the different schools of thought on what makes an explanation, and suggest that machine learning might benefit from viewing the problem more broadly.
---
paper_title: YASENN: Explaining Neural Networks via Partitioning Activation Sequences

paper_content:

We introduce a novel approach to feed-forward neural network interpretation based on partitioning the space of sequences of neuron activations. In line with this approach, we propose a model-specific interpretation method, called YASENN. Our method inherits many advantages of model-agnostic distillation, such as an ability to focus on the particular input region and to express an explanation in terms of features different from those observed by a neural network. Moreover, examination of distillation error makes the method applicable to the problems with low tolerance to interpretation mistakes. Technically, YASENN distills the network with an ensemble of layer-wise gradient boosting decision trees and encodes the sequences of neuron activations with leaf indices. The finite number of unique codes induces a partitioning of the input space. Each partition may be described in a variety of ways, including examination of an interpretable model (e.g. a logistic regression or a decision tree) trained to discriminate between objects of those partitions. Our experiments provide an intuition behind the method and demonstrate revealed artifacts in neural network decision making.
---
paper_title: Towards A Rigorous Science of Interpretable Machine Learning

paper_content:

As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.
---
paper_title: A Case-Based Explanation System for Black-Box Systems

paper_content:

Most users of machine-learning products are reluctant to use them without any sense of the underlying logic that has led to the system's predictions. Unfortunately many of these systems lack any transparency in the way they operate and are deemed to be black boxes. In this paper we present a Case-Based Reasoning (CBR) solution to providing supporting explanations of black-box systems. This CBR solution has two key facets; it uses local information to assess the importance of each feature and using this, it selects the cases from the data used to build the black-box system for use in explanation. The retrieval mechanism takes advantage of the derived feature importance information to help select cases that are a better reflection of the black-box solution and thus more convincing explanations.
---
paper_title: Gaining insight through case-based explanation

paper_content:

Traditional explanation strategies in machine learning have been dominated by rule and decision tree based approaches. Case-based explanations represent an alternative approach which has inherent advantages in terms of transparency and user acceptability. Case-based explanations are based on a strategy of presenting similar past examples in support of and as justification for recommendations made. The traditional approach to such explanations, of simply supplying the nearest neighbour as an explanation, has been found to have shortcomings. Cases should be selected based on their utility in forming useful explanations. However, the relevance of the explanation case may not be clear to the end user as it is retrieved using domain knowledge which they themselves may not have. In this paper the focus is on a knowledge-light approach to case-based explanations that works by selecting cases based on explanation utility and offering insights into the effects of feature-value differences. In this paper we examine to two such a knowledge-light frameworks for case-based explanation. We look at explanation oriented retrieval (EOR) a strategy which explicitly models explanation utility and also at the knowledge-light explanation framework (KLEF) that uses local logistic regression to support case-based explanation.
---
paper_title: Case-Based Reasoning for Explaining Probabilistic Machine Learning

paper_content:

This paper describes a generic framework for explaining the prediction of probabilistic machine learning algorithms using cases. The framework consists of two components: a similarity metric between cases that is defined relative to a probability model and an novel case-based approach to justifying the probabilistic prediction by estimating the prediction error using case-based reasoning. As basis for deriving similarity metrics, we define similarity in terms of the principle of interchangeability that two cases are considered similar or identical if two probability distributions, derived from excluding either one or the other case in the case base, are identical. Lastly, we show the applicability of the proposed approach by deriving a metric for linear regression, and apply the proposed approach for explaining predictions of the energy performance of households.
---
paper_title: The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification

paper_content:

We present the Bayesian Case Model (BCM), a general framework for Bayesian case-based reasoning (CBR) and prototype classification and clustering. BCM brings the intuitive power of CBR to a Bayesian generative framework. The BCM learns prototypes, the "quintessential" observations that best represent clusters in a dataset, by performing joint inference on cluster labels, prototypes and important features. Simultaneously, BCM pursues sparsity by learning subspaces, the sets of features that play important roles in the characterization of the prototypes. The prototype and subspace representation provides quantitative benefits in interpretability while preserving classification accuracy. Human subject experiments verify statistically significant improvements to participants' understanding when using explanations produced by BCM, compared to those given by prior art.
---
paper_title: An Evaluation of the Usefulness of Case-Based Explanation

paper_content:

One of the perceived benefits of Case-Based Reasoning (CBR) is the potential to use retrieved cases to explain predictions. Surprisingly, this aspect of CBR has not been much researched. There has been some early work on knowledge-intensive approaches to CBR where the cases contain explanation patterns (e.g. SWALE). However, a more knowledge-light approach where the case similarity is the basis for explanation has received little attention. To explore this, we have developed a CBR system for predicting blood-alcohol level. We compare explanations of predictions produced by this system with alternative rule-based explanations. The case-based explanations fare very well in this evaluation and score significantly better than the rule-based alternative.
---
paper_title: DeepRED – Rule Extraction from Deep Neural Networks

paper_content:

Neural network classifiers are known to be able to learn very accurate models. In the recent past, researchers have even been able to train neural networks with multiple hidden layers (deep neural networks) more effectively and efficiently. However, the major downside of neural networks is that it is not trivial to understand the way how they derive their classification decisions. To solve this problem, there has been research on extracting better understandable rules from neural networks. However, most authors focus on nets with only one single hidden layer. The present paper introduces a new decompositional algorithm – DeepRED – that is able to extract rules from deep neural networks.
---
paper_title: Massively Parallel Case-Based Reasoning with Probabilistic Similarity Metrics

paper_content:

We propose a probabilistic case-space metric for the case matching and case adaptation tasks. Central to our approach is a probability propagation algorithm adopted from Bayesian reasoning systems, which allows our case-based reasoning system to perform theoretically sound probabilistic reasoning. The same probability propagation mechanism actually offers a uniform solution to both the case matching and case adaptation problems. We also show how the algorithm can be implemented as a connectionist network, where efficient massively parallel case retrieval is an inherent property of the system. We argue that using this kind of an approach, the difficult problem of case indexing can be completely avoided.
---
paper_title: On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

paper_content:

Understanding and interpreting classification decisions of automated image classification systems is of high value in many applications, as it allows to verify the reasoning of the system and provides additional information to the human expert. Although machine learning methods are solving very successfully a plethora of tasks, they have in most cases the disadvantage of acting as a black box, not providing any information about what made them arrive at a particular decision. This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers. We introduce a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks. These pixel contributions can be visualized as heatmaps and are provided to a human expert who can intuitively not only verify the validity of the classification decision, but also focus further analysis on regions of potential interest. We evaluate our method for classifiers trained on PASCAL VOC 2009 images, synthetic image data containing geometric shapes, the MNIST handwritten digits data set and for the pre-trained ImageNet model available as part of the Caffe open source package.
---
paper_title: In Defense of Fully Connected Layers in Visual Representation Transfer

paper_content:

Pre-trained convolutional neural network (CNN) models have been widely applied in many computer vision tasks, especially in transfer learning tasks. In transfer learning, the target domain may be in a different feature space or follow a different data distribution, compared to the source domain. In CNN transfer tasks, we often transfer visual representations from a source domain (e.g., ImageNet) to target domains with fewer training images or have different image properties. It is natural to explore which CNN model performs better in visual representation transfer. Through visualization analyses and extensive experiments, we show that when either image properties or task objective in the target domain is far away from those in the source domain, having the fully connected layers in the source domain pre-trained model is essential in achieving high accuracy after transferring to the target domain.
---
paper_title: Deep Weighted Averaging Classifiers

paper_content:

Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to modify any conventional deep architecture to automatically provide more transparent explanations for classification decisions, as well as an intuitive notion of the credibility of each prediction. Specifically, we draw on ideas from nonparametric kernel regression, and propose to predict labels based on a weighted sum of training instances, where the weights are determined by distance in a learned instance-embedding space. Working within the framework of conformal methods, we propose a new measure of nonconformity suggested by our model, and experimentally validate the accompanying theoretical expectations, demonstrating improved transparency, controlled error rates, and robustness to out-of-domain data, without compromising on accuracy or calibration.
---
paper_title: The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification

paper_content:

We present the Bayesian Case Model (BCM), a general framework for Bayesian case-based reasoning (CBR) and prototype classification and clustering. BCM brings the intuitive power of CBR to a Bayesian generative framework. The BCM learns prototypes, the "quintessential" observations that best represent clusters in a dataset, by performing joint inference on cluster labels, prototypes and important features. Simultaneously, BCM pursues sparsity by learning subspaces, the sets of features that play important roles in the characterization of the prototypes. The prototype and subspace representation provides quantitative benefits in interpretability while preserving classification accuracy. Human subject experiments verify statistically significant improvements to participants' understanding when using explanations produced by BCM, compared to those given by prior art.
---
paper_title: Axiomatic Attribution for Deep Networks

paper_content:

We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms— Sensitivity and Implementation Invariance that attribution methods ought to satisfy. We show that they are not satisfied by most known attribution methods, which we consider to be a fundamental weakness of those methods. We use the axioms to guide the design of a new attribution method called Integrated Gradients. Our method requires no modification to the original network and is extremely simple to implement; it just needs a few calls to the standard gradient operator. We apply this method to a couple of image models, a couple of text models and a chemistry model, demonstrating its ability to debug networks, to extract rules from a network, and to enable users to engage with models better.
---
paper_title: Learning Important Features Through Propagating Activation Differences

paper_content:

The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. Video tutorial: http://goo.gl/qKb7pL, code: http://goo.gl/RM8jvH.
---
paper_title: Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning

paper_content:

Deep neural networks (DNNs) enable innovative applications of machine learning like image recognition, machine translation, or malware detection. However, deep learning is often criticized for its lack of robustness in adversarial settings (e.g., vulnerability to adversarial inputs) and general inability to rationalize its predictions. In this work, we exploit the structure of deep learning to enable new learning-based inference and decision strategies that achieve desirable properties such as robustness and interpretability. We take a first step in this direction and introduce the Deep k-Nearest Neighbors (DkNN). This hybrid classifier combines the k-nearest neighbors algorithm with representations of the data learned by each layer of the DNN: a test input is compared to its neighboring training points according to the distance that separates them in the representations. We show the labels of these neighboring points afford confidence estimates for inputs outside the model's training manifold, including on malicious inputs like adversarial examples--and therein provides protections against inputs that are outside the models understanding. This is because the nearest neighbors can be used to estimate the nonconformity of, i.e., the lack of support for, a prediction in the training data. The neighbors also constitute human-interpretable explanations of predictions. We evaluate the DkNN algorithm on several datasets, and show the confidence estimates accurately identify inputs outside the model, and that the explanations provided by nearest neighbors are intuitive and useful in understanding model failures.
---
paper_title: This Looks Like That: Deep Learning for Interpretable Image Recognition

paper_content:

When we are faced with challenging image classification tasks, we often explain our reasoning by dissecting the image, and pointing out prototypical aspects of one class or another. The mounting evidence for each of the classes helps us make our final decision. In this work, we introduce a deep network architecture -- prototypical part network (ProtoPNet), that reasons in a similar way: the network dissects the image by finding prototypical parts, and combines evidence from the prototypes to make a final classification. The model thus reasons in a way that is qualitatively similar to the way ornithologists, physicians, and others would explain to people on how to solve challenging image classification tasks. The network uses only image-level labels for training without any annotations for parts of images. We demonstrate our method on the CUB-200-2011 dataset and the Stanford Cars dataset. Our experiments show that ProtoPNet can achieve comparable accuracy with its analogous non-interpretable counterpart, and when several ProtoPNets are combined into a larger network, it can achieve an accuracy that is on par with some of the best-performing deep models. Moreover, ProtoPNet provides a level of interpretability that is absent in other interpretable deep models.
---
paper_title: Deep Learning for Case-based Reasoning through Prototypes: A Neural Network that Explains its Predictions

paper_content:

Deep neural networks are widely used for classification. These deep models often suffer from a lack of interpretability -- they are particularly difficult to understand because of their non-linear nature. As a result, neural networks are often treated as "black box" models, and in the past, have been trained purely to optimize the accuracy of predictions. In this work, we create a novel network architecture for deep learning that naturally explains its own reasoning for each prediction. This architecture contains an autoencoder and a special prototype layer, where each unit of that layer stores a weight vector that resembles an encoded training input. The encoder of the autoencoder allows us to do comparisons within the latent space, while the decoder allows us to visualize the learned prototypes. The training objective has four terms: an accuracy term, a term that encourages every prototype to be similar to at least one encoded input, a term that encourages every encoded input to be close to at least one prototype, and a term that encourages faithful reconstruction by the autoencoder. The distances computed in the prototype layer are used as part of the classification process. Since the prototypes are learned during training, the learned network naturally comes with explanations for each prediction, and the explanations are loyal to what the network actually computes.
---
paper_title: A Survey of Methods for Explaining Black Box Models

paper_content:

In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.
---

```
Title: How Case Based Reasoning Explained Neural Networks: An XAI Survey of Post-Hoc Explanation-by-Example in ANN-CBR Twins

Section 1: Introduction
Description 1: Provide an overview of the importance of explainable AI (XAI), explain the twin-systems approach, and introduce the concept of ANN-CBR twins.

Section 2: "Explanation" Needs Explanation
Description 2: Discuss the various interpretations and definitions of "explanation" within the context of XAI and how CBR systems can provide post-hoc explanation-by-example.

Section 3: Motivation for a Systematic Review
Description 3: Explain the reasons behind conducting a systematic review of ANN-CBR twins, highlighting gaps and fragmentation in the current literature.

Section 4: Defining ANN-CBR Twins
Description 4: Define ANN-CBR twin-systems, detailing the specific criteria and components that distinguish them from other hybrid systems.

Section 5: A Systematic Review: Methodology
Description 5: Describe the search procedures, including the systematic top-down searches and bottom-up citation-based searches, used to gather relevant literature on ANN-CBR twins.

Section 6: Results Summary
Description 6: Present an overview of the findings from the systematic search, including the number of papers reviewed and the final selection of relevant studies.

Section 7: A History of ANN-CBR Twins
Description 7: Outline the historical development of ANN-CBR twins, identifying key periods and contributions from different research groups.

Section 8: Korean Developments (1999-2007): Feature-Weighting Tests of Twins
Description 8: Discuss the significant contributions and findings from the South Korean group at KAIST, focusing on their comparative tests of feature-weighting techniques.

Section 9: A Parallel Discovery in L.A.: Caruana et al. (1999)
Description 9: Provide an overview of Caruana et al.'s work on using MLPs to provide case-based explanations, highlighting differences from the Korean approach.

Section 10: An Irish Departure (mid-2000s): Local Feature-Weighting Tests of Twins
Description 10: Discuss the contributions from the Irish group at University College Dublin, focusing on their local feature-weighting methods and user tests.

Section 11: Recent DNN-CBR Twinning
Description 11: Explore recent developments and approaches for combining DNNs with CBR for explainability, identifying notable techniques and their applications.

Section 12: Future Directions: Road Mapping
Description 12: Summarize the significance of the survey and propose future research directions for ANN-CBR twins in addressing XAI challenges.
```