Efficient Rehearsal Free Zero Forgetting Continual Learning using
  Adaptive Weight Modulation
Yonatan Sverdlov Shimon Ullman
Artificial neural networks encounter a notable challenge known as continual
learning, which involves acquiring knowledge of multiple tasks over an extended
period. This challenge arises due to the tendency of previously learned weights
to be adjusted to suit the objectives of new tasks, resulting in a phenomenon
called catastrophic forgetting. Most approaches to this problem seek a balance
between maximizing performance on the new tasks and minimizing the forgetting
of previous tasks. In contrast, our approach attempts to maximize the
performance of the new task, while ensuring zero forgetting. This is
accomplished by creating a task-specific modulation parameters for each task.
Only these would be learnable parameters during learning of consecutive tasks.
Through comprehensive experimental evaluations, our model demonstrates superior
performance in acquiring and retaining novel tasks that pose difficulties for
other multi-task models. This emphasizes the efficacy of our approach in
preventing catastrophic forgetting while accommodating the acquisition of new
tasks