Recall-Oriented Continual Learning with Generative Adversarial
  Meta-Model
Haneol Kang Dong-Wan Choi
The stability-plasticity dilemma is a major challenge in continual learning,
as it involves balancing the conflicting objectives of maintaining performance
on previous tasks while learning new tasks. In this paper, we propose the
recall-oriented continual learning framework to address this challenge.
Inspired by the human brain's ability to separate the mechanisms responsible
for stability and plasticity, our framework consists of a two-level
architecture where an inference network effectively acquires new knowledge and
a generative network recalls past knowledge when necessary. In particular, to
maximize the stability of past knowledge, we investigate the complexity of
knowledge depending on different representations, and thereby introducing
generative adversarial meta-model (GAMM) that incrementally learns
task-specific parameters instead of input data samples of the task. Through our
experiments, we show that our framework not only effectively learns new
knowledge without any disruption but also achieves high stability of previous
knowledge in both task-aware and task-agnostic learning scenarios. Our code is
available at: https://github.com/bigdata-inha/recall-oriented-cl-framework.