We present a framework for interpretable continual learning (ICL). We show that explanations of previously performed tasks can be used to improve performance on future tasks. ICL generates a good explanation of a finished task, then uses this to focus attention on what is important when facing a new task. The ICL idea is general and may be applied to many continual learning approaches. Here we focus on the variational continual learning framework to take advantage of its flexibility and efficacy in overcoming catastrophic forgetting. We use saliency maps to provide explanations of performed tasks and propose a new metric to assess their quality. Experiments show that ICL achieves state-of-the-art results in terms of overall continual learning performance as measured by average classification accuracy, and also in terms of its explanations, which are assessed qualitatively and quantitatively using the proposed metric. Continual learning, also called lifelong learning, refers to frameworks where knowledge acquired from past tasks is accumulated for use on future tasks, i.e. where learning continually proceeds in an online fashion. Data belonging to different tasks might be non i.i.d. BID17 BID20 BID25 BID24 BID29 . Crucially, a continual learner must be able to learn a new task without forgetting how to perform previous tasks BID19 BID27 . Continual learning frameworks need to continually adapt to the domain shift occurring across tasks, without revisiting data from previous tasks. An appropriate balance is required between stability and adapting to new tasks and data, since excessive adaptation might lead to dramatic degradation in performance of earlier tasks, known as catastrophic forgetting BID6 BID7 BID16 BID18 .Several approaches have been introduced to address catastrophic forgetting. One approach is based on regularisation where stability is maintained by restricting the change of parameters with high influence while allowing the other parameters to vary freely BID14 BID17 BID11 BID30 BID35 . Another approach divides the network architecture into reusable parts that are less prone to changes, and parts devoted to individual tasks BID4 BID23 a; BID34 . The framework in BID33 constructs the neural network architecture via designed reinforcement learning (RL) strategies. The framework in BID13 bases its solution on moment matching. Another RL based framework is presented in BID9 where catastrophic forgetting is mitigated at multiple time scales by using RL agents with a synaptic model. BID15 propose a framework where multiple agents jointly learn to achieve multiple goals at once in a parallel offpolicy setup. The work in BID3 proposes experimental evaluations of continual learning as well as a variational Bayesian loss, via which they categorise a few previous works into either prior-focused or likelihood-focused. Attention mechanisms have been developed in rather similar problems before, e.g. in BID10 BID28 BID31 BID8 . Other saliency metrics have been introduced in BID2 BID0 . To the best of our knowledge , our framework is the first piece of work to pursue a comprehensive interpretability approach in the continual learning setting.Our work is based on the idea of imitating some aspects of how humans learn continually. We suggest that humans are quite successful in achieving goals and performing tasks sequentially, partly because we manage to understand and explain to ourselves certain aspects of the tasks we have already accomplished. This provides a contribution to our performance on similar tasks in the future.When given a task, we typically not only accomplish it, but also often (perhaps unconsciously) gain a useful interpretable concept. For instance, when a child tries to grasp an object, the progress in the child's skill is accompanied by improvements in similar tasks, e.g. grasping other objects which the child has never seen before. The development of the child's cognitive abilities shows an understanding of the concepts of gravity, geometric characteristics of the object, etc BID26 . We suggest that this development may be established by the consolidation of interpretable information from past experience. With this motivation, here we consider and analyse the impact of interpretable methods on continual learning. While there can be a tradeoff between performance and interpretability when considering just one task, here we show that interpretability can help performance when a learner faces consecutive tasks over time, as in continual learning.We propose a continual learning framework where the training phase of each task is followed by an explanation stage, which provides insights to be utilised along with the established training platform in learning the subsequent tasks. We focus on image classification tasks where training of the first task proceeds normally, before providing a saliency map for each test data point (image). By highlighting the most relevant areas of the test image w.r.t. the classification prediction, an understanding of the individual decisions taken by the classifier is attained. Afterwards, we compute a saliency map representing a summary (or average) of the saliency maps of test images per class, i.e. a summary of the classifier's decisions on the test data. This represents a summarised belief of the relevant areas of the input for each class. After completing the current task, we can use the explanation , depicted by the average saliency map, to achieve two goals:i Assessing how good the developed continual learning framework is at eradicating catastrophic forgetting. This evaluation is typically performed by measuring the difference in the classification performance on a certain task when it is the most recently encountered vs. when other tasks subsequently followed. Here we propose a new measure by adding a test which compares the saliency map resulting from testing a task right after finishing the corresponding training phase, with the saliency map of the same test on the same task after having other tasks subsequently trained by our continual learner. Degradation of the provided explanations provides a measure of the level of catastrophic forgetting induced in the continual learning framework. ii Providing interpretable attention information for subsequent tasks by involving the obtained saliency maps of the current task in the optimisation for the future tasks.To achieve the first goal, we need a metric to assess the quality of the extracted saliency maps. We propose a new metric for evaluating saliency maps resulting from the classification decisions on test data. Measuring the quality of a saliency map is not a straightforward task. Saliency maps typically aim at explaining the classification decision taken by a classifier. In other words, a saliency map seeks the subset of features that are the most influential in the resulting prediction of the classifier. As such, the explanation provided by a saliency map comes out in the form of a summary of the significant parts (features) of the input data, from the classifier's point of view. We propose a metric to assess the quality of an explanation resulting from a saliency map.To achieve the second goal, we need to involve the saliency maps in the learning procedure, not just during the test phase. We propose an attention mechanism that exploits the feature relevance values learnt in the latest task to focus the attention of the new learner on what is believed to be the most important parts of the input as per the latest task, which (w.r.t. continual learning) is assumed to be similar. Thus, an explanation of a task evolving over time is utilised to help the emerging tasks.Note that there is a difference in the nature of the two proposed goals described above. The optimisation needed to achieve the second goal does not guarantee that the first goal is automatically achieved, since the first goal addresses explanations (saliency maps) related to the same task at different time steps, whereas the second goal is concerned with exchanging interpretable information among different tasks. Hence, the assessment involved in the first goal is still needed regardless of the level of perfection of the second goal.In this work we perform experiments for the classification case illustrated above, but the same paradigm could be applied in future work to tasks other than classification, where the personalisation or notion of the explanation will need to be adapted to the nature of the task, i.e. saliency maps should be replaced with something more suitable to the task.We highlight the following contributions of our framework: 1) the interpretable continual learning (ICL) framework where, in addition to the ordinary understanding benefits of interpretable frame-works, explanations of the finished tasks are used to enhance the attention of the learner during the future tasks. Although we focus on performing within the variational continual learning framework BID17 , our proposed methodology is flexible and can be deployed with other continual learning frameworks, and also with other saliency detection methods. To the best of our knowledge, ICL represents a novel direction in the continual learning literature that focuses on interpretability; 2) introducing a new metric for saliency maps that aims at robustly assessing their quality without significant engineering required, i.e. no need to construct bounding rectangles of the relevant zones or similar; 3) learning from past experiences using an attention mechanism based on explanations (saliency maps) from the latest task; 4) Our quantitative and qualitative state-of-the-art results in four experiments on three datasets demonstrate the efficacy of the proposed framework. We introduced a continual learning framework incorporating interpretability, where saliency based explanations of previously learnt tasks are used to enhance the attention of the learner during future tasks. This framework demonstrates that interpretability is not only useful for increasing the understanding of the obtained results, but can also improve the performance of a sequential learning procedure. The proposed framework is flexible and can enhance both the interpretability and performance of continual learning methods, especially in terms of mitigating catastrophic forgetting. We proposed a new metric for saliency maps. We believe that adopting a Bayesian attention mechanism could be a fruitful direction for future work, especially when integrated with fully Bayesian variational continual learning.
The paper develops an interpretable continual learning framework where explanations of the finished tasks are used to enhance the attention of the learner during the future tasks, and where an explanation metric is proposed too.  The authors propose a framework for continual learning based on explanations for performed classifications of previously learned tasks This paper proposes an extension to the continual learning framework using existing variational continual learning as the base method with weight of evidence.