Improving Forward Compatibility in Class Incremental Learning by
  Increasing Representation Rank and Feature Richness
Jaeill Kim Wonseok Lee Moonjung Eo Wonjong Rhee
Class Incremental Learning (CIL) constitutes a pivotal subfield within
continual learning, aimed at enabling models to progressively learn new
classification tasks while retaining knowledge obtained from prior tasks.
Although previous studies have predominantly focused on backward compatible
approaches to mitigate catastrophic forgetting, recent investigations have
introduced forward compatible methods to enhance performance on novel tasks and
complement existing backward compatible methods. In this study, we introduce an
effective-Rank based Feature Richness enhancement (RFR) method, designed for
improving forward compatibility. Specifically, this method increases the
effective rank of representations during the base session, thereby facilitating
the incorporation of more informative features pertinent to unseen novel tasks.
Consequently, RFR achieves dual objectives in backward and forward
compatibility: minimizing feature extractor modifications and enhancing novel
task performance, respectively. To validate the efficacy of our approach, we
establish a theoretical connection between effective rank and the Shannon
entropy of representations. Subsequently, we conduct comprehensive experiments
by integrating RFR into eleven well-known CIL methods. Our results demonstrate
the effectiveness of our approach in enhancing novel-task performance while
mitigating catastrophic forgetting. Furthermore, our method notably improves
the average incremental accuracy across all eleven cases examined.