Efficient Expansion and Gradient Based Task Inference for Replay Free
  Incremental Learning
Soumya Roy Vinay K Verma Deepak Gupta
This paper proposes a simple but highly efficient expansion-based model for
continual learning. The recent feature transformation, masking and
factorization-based methods are efficient, but they grow the model only over
the global or shared parameter. Therefore, these approaches do not fully
utilize the previously learned information because the same task-specific
parameter forgets the earlier knowledge. Thus, these approaches show limited
transfer learning ability. Moreover, most of these models have constant
parameter growth for all tasks, irrespective of the task complexity. Our work
proposes a simple filter and channel expansion based method that grows the
model over the previous task parameters and not just over the global parameter.
Therefore, it fully utilizes all the previously learned information without
forgetting, which results in better knowledge transfer. The growth rate in our
proposed model is a function of task complexity; therefore for a simple task,
the model has a smaller parameter growth while for complex tasks, the model
requires more parameters to adapt to the current task. Recent expansion based
models show promising results for task incremental learning (TIL). However, for
class incremental learning (CIL), prediction of task id is a crucial challenge;
hence, their results degrade rapidly as the number of tasks increase. In this
work, we propose a robust task prediction method that leverages entropy
weighted data augmentations and the models gradient using pseudo labels. We
evaluate our model on various datasets and architectures in the TIL, CIL and
generative continual learning settings. The proposed approach shows
state-of-the-art results in all these settings. Our extensive ablation studies
show the efficacy of the proposed components.