A Bag of Tricks for Few-Shot Class-Incremental Learning
Shuvendu Roy Chunjong Park Aldi Fahrezi Ali Etemad
We present a bag of tricks framework for few-shot class-incremental learning
(FSCIL), which is a challenging form of continual learning that involves
continuous adaptation to new tasks with limited samples. FSCIL requires both
stability and adaptability, i.e., preserving proficiency in previously learned
tasks while learning new ones. Our proposed bag of tricks brings together eight
key and highly influential techniques that improve stability, adaptability, and
overall performance under a unified framework for FSCIL. We organize these
tricks into three categories: stability tricks, adaptability tricks, and
training tricks. Stability tricks aim to mitigate the forgetting of previously
learned classes by enhancing the separation between the embeddings of learned
classes and minimizing interference when learning new ones. On the other hand,
adaptability tricks focus on the effective learning of new classes. Finally,
training tricks improve the overall performance without compromising stability
or adaptability. We perform extensive experiments on three benchmark datasets,
CIFAR-100, CUB-200, and miniIMageNet, to evaluate the impact of our proposed
framework. Our detailed analysis shows that our approach substantially improves
both stability and adaptability, establishing a new state-of-the-art by
outperforming prior works in the area. We believe our method provides a go-to
solution and establishes a robust baseline for future research in this area.