Realistic Continual Learning Approach using Pre-trained Models
Nadia Nasri Carlos Gutiérrez-Álvarez Sergio Lafuente-Arroyo Saturnino Maldonado-Bascón Roberto J. López-Sastre
Continual learning (CL) is crucial for evaluating adaptability in learning
solutions to retain knowledge. Our research addresses the challenge of
catastrophic forgetting, where models lose proficiency in previously learned
tasks as they acquire new ones. While numerous solutions have been proposed,
existing experimental setups often rely on idealized class-incremental learning
scenarios. We introduce Realistic Continual Learning (RealCL), a novel CL
paradigm where class distributions across tasks are random, departing from
structured setups.
  We also present CLARE (Continual Learning Approach with pRE-trained models
for RealCL scenarios), a pre-trained model-based solution designed to integrate
new knowledge while preserving past learning. Our contributions include
pioneering RealCL as a generalization of traditional CL setups, proposing CLARE
as an adaptable approach for RealCL tasks, and conducting extensive experiments
demonstrating its effectiveness across various RealCL scenarios. Notably, CLARE
outperforms existing models on RealCL benchmarks, highlighting its versatility
and robustness in unpredictable learning environments.