Neural Collapse Terminus: A Unified Solution for Class Incremental
  Learning and Its Variants
Yibo Yang Haobo Yuan Xiangtai Li Jianlong Wu Lefei Zhang Zhouchen Lin Philip Torr Dacheng Tao Bernard Ghanem
How to enable learnability for new classes while keeping the capability well
on old classes has been a crucial challenge for class incremental learning.
Beyond the normal case, long-tail class incremental learning and few-shot class
incremental learning are also proposed to consider the data imbalance and data
scarcity, respectively, which are common in real-world implementations and
further exacerbate the well-known problem of catastrophic forgetting. Existing
methods are specifically proposed for one of the three tasks. In this paper, we
offer a unified solution to the misalignment dilemma in the three tasks.
Concretely, we propose neural collapse terminus that is a fixed structure with
the maximal equiangular inter-class separation for the whole label space. It
serves as a consistent target throughout the incremental training to avoid
dividing the feature space incrementally. For CIL and LTCIL, we further propose
a prototype evolving scheme to drive the backbone features into our neural
collapse terminus smoothly. Our method also works for FSCIL with only minor
adaptations. Theoretical analysis indicates that our method holds the neural
collapse optimality in an incremental fashion regardless of data imbalance or
data scarcity. We also design a generalized case where we do not know the total
number of classes and whether the data distribution is normal, long-tail, or
few-shot for each coming session, to test the generalizability of our method.
Extensive experiments with multiple datasets are conducted to demonstrate the
effectiveness of our unified solution to all the three tasks and the
generalized case.