In representational lifelong learning an agent aims to continually learn to solve novel tasks while updating its representation in light of previous tasks. Under the assumption that future tasks are related to previous tasks, representations should be learned in such a way that they capture the common structure across learned tasks, while allowing the learner sufficient flexibility to adapt to novel aspects of a new task. We develop a framework for lifelong learning in deep neural networks that is based on generalization bounds, developed within the PAC-Bayes framework. Learning takes place through the construction of a distribution over networks based on the tasks seen so far, and its utilization for learning a new task. Thus, prior knowledge is incorporated through setting a history-dependent prior for novel tasks. We develop a gradient-based algorithm implementing these ideas, based on minimizing an objective function motivated by generalization bounds, and demonstrate its effectiveness through numerical examples. Learning from examples is the process of inferring a general rule from a finite set of examples. It is well known in statistics (e.g., BID7 ) that learning cannot take place without prior assumptions. This idea has led in Machine Learning to the notion of inductive bias BID23 . Recent work in deep neural networks has achieved significant success in using prior knowledge in the implementation of structural constraints, e.g. the use of convolutions and weight sharing as building blocks, capturing the translational invariance of image classification. However, in general the relevant prior information for a given task is not always clear, and there is a need for building prior knowledge through learning from previous interactions with the world.Learning from previous experience can take several forms: Continual learning -a single model is trained to solve a task which changes over time (and hopefully not 'forget' the knowledge from previous times, (e.g., ). Multi-task learning -the goal is to learn how to solve several observed tasks, while exploiting their shared structure. Domain adaptation -the goal is to solve a 'target' learning task using a single 'source' learning task (both are observed, but usually the target has mainly unlabeled data). Lifelong Learning / Meta-Learning / Learning-to-Learnthe goal is to extract knowledge from several observed tasks to be used for future learning on new (not yet observed) learning tasks. In contrast to multi-task learning, the performance is evaluated on the new tasks.We work within the framework of lifelong learning, where an agent learns through interacting with the world, transferring the knowledge acquired along its path to any new task it encounters. This notion has been formulated by BID3 in a clear and simple context of 'task-environment'. In analogy to the standard single-task learning in which data is sampled from an unknown distribution, Baxter suggested to model a lifelong learning setting as if tasks are sampled from an unknown task distribution (environment), so that knowledge acquired from previous tasks can be used in order to improve performance on a novel task. Baxter's work not only provided an interesting and mathematically precise perspective for lifelong learning, but also provided generalization bounds demonstrating the potential improvement in performance due to prior knowledge. Baxter's seminal work, has led to a large number of extensions and developments.In this contribution we work within the framework formulated by BID3 , and, following the setup in BID25 , provide generalization error bounds within the PAC-Bayes framework. These bounds are then used to develop a practical learning algorithm that is applied to neural networks, demonstrating the utility of the approach. The main contributions of this work are the following. (i) An improved and tighter bound in the theoretical framework of BID25 which can utilize different single-task PAC-Bayesian bounds.(ii ) Developing a learning algorithm within this general framework and its implementation using probabilistic feedforward neural networks. This yields transfer of knowledge between tasks through constraining the prior distribution on a learning network. (iii ) Empirical demonstration of the performance enhancement compared to naive approaches and recent methods in this field.As noted above, BID3 provided a basic mathematical formulation and initial results for lifelong learning. While there have been many developments in this field since then (e.g., BID1 ; BID9 BID10 ; BID27 ), most of them were not based on generalization error bounds which is the focus of the present work. An elegant extension of generalization error bounds to lifelong learning was provided by BID25 , mentioned above (more recently extended in BID26 ). Their work , however, did not provide a practical algorithm applicable to deep neural networks. More recently , Dziugaite & Roy (2017) developed a single-task algorithm based on PAC-Bayes bounds that was demonstrated to yield good performance in simple classification tasks. Other recent theoretical approaches to lifelong or multitask learning (e.g. BID0 ; BID20 ) provide increasingly general bounds but have not led directly to practical learning algorithms. We have presented a framework for representational lifelong learning, motivated by PAC-Bayes generalization bounds, and implemented through the adjustment of a learned prior, based on tasks encountered so far. The framework bears conceptual similarity to the empirical Bayes method while not being Bayesian, and is implemented at the level of tasks rather than samples. Combining the general approach with the rich representational structure of deep neural networks, and learning through gradient based methods leads to an efficient procedure for lifelong learning, as motivated theoretically and demonstrated empirically. While our experimental results are preliminary, we believe that our work attests to the utility of using rigorous performance bounds to derive learning algorithms, and demonstrates that tighter bounds indeed lead to improved performance.There are several open issues to consider. First, the current version learns to solve all available tasks in parallel, while a more useful procedure should be sequential in nature. This can be easily incorporated into our framework by updating the prior following each novel task. Second, our method requires training stochastic models which is challenging due to the the high-variance gradients. We we would like to develop new methods within our framework which have more stable convergence and are easier to apply in larger scale problems. Third, there is much current effort in reinforcement learning to augment model free learning with model based components, where some aspects of the latter are often formulated as supervised learning tasks. Incorporating our approach in such a context would be a worthwhile challenge. In fact, a similar framework to ours was recently proposed within an RL setting BID33 , although it was not motivated from performance guarantees as was our approach, but rather from intuitive heuristic arguments.
We develop a lifelong learning approach to transfer learning based on PAC-Bayes theory, whereby priors are adjusted as new tasks are encountered thereby facilitating the learning of novel tasks. A novel PAC-Bayesian risk bound that serves as an objective function for multi-task machine learning, and an algorithm for minimizing a simplified version of that objective function. Extends existing PAC-Bayes bounds to multi-task learning, to allow the prior to be adapted across different tasks.