Federated Class-Incremental Learning with Prototype Guided Transformer
Haiyang Guo Fei Zhu Wenzhuo Liu Xu-Yao Zhang Cheng-Lin Liu
Existing federated learning methods have effectively addressed decentralized
learning in scenarios involving data privacy and non-IID data. However, in
real-world situations, each client dynamically learns new classes, requiring
the global model to maintain discriminative capabilities for both new and old
classes. To effectively mitigate the effects of catastrophic forgetting and
data heterogeneity under low communication costs, we designed a simple and
effective method named PLoRA. On the one hand, we adopt prototype learning to
learn better feature representations and leverage the heuristic information
between prototypes and class features to design a prototype re-weight module to
solve the classifier bias caused by data heterogeneity without retraining the
classification layer. On the other hand, our approach utilizes a pre-trained
model as the backbone and utilizes LoRA to fine-tune with a tiny amount of
parameters when learning new classes. Moreover, PLoRA does not rely on
similarity-based module selection strategies, thereby further reducing
communication overhead. Experimental results on standard datasets indicate that
our method outperforms the state-of-the-art approaches significantly. More
importantly, our method exhibits strong robustness and superiority in various
scenarios and degrees of data heterogeneity. Our code will be publicly
available.