Text-Enhanced Data-free Approach for Federated Class-Incremental
  Learning
Minh-Tuan Tran Trung Le Xuan-May Le Mehrtash Harandi Dinh Phung
Federated Class-Incremental Learning (FCIL) is an underexplored yet pivotal
issue, involving the dynamic addition of new classes in the context of
federated learning. In this field, Data-Free Knowledge Transfer (DFKT) plays a
crucial role in addressing catastrophic forgetting and data privacy problems.
However, prior approaches lack the crucial synergy between DFKT and the model
training phases, causing DFKT to encounter difficulties in generating
high-quality data from a non-anchored latent space of the old task model. In
this paper, we introduce LANDER (Label Text Centered Data-Free Knowledge
Transfer) to address this issue by utilizing label text embeddings (LTE)
produced by pretrained language models. Specifically, during the model training
phase, our approach treats LTE as anchor points and constrains the feature
embeddings of corresponding training samples around them, enriching the
surrounding area with more meaningful information. In the DFKT phase, by using
these LTE anchors, LANDER can synthesize more meaningful samples, thereby
effectively addressing the forgetting problem. Additionally, instead of tightly
constraining embeddings toward the anchor, the Bounding Loss is introduced to
encourage sample embeddings to remain flexible within a defined radius. This
approach preserves the natural differences in sample embeddings and mitigates
the embedding overlap caused by heterogeneous federated settings. Extensive
experiments conducted on CIFAR100, Tiny-ImageNet, and ImageNet demonstrate that
LANDER significantly outperforms previous methods and achieves state-of-the-art
performance in FCIL. The code is available at
https://github.com/tmtuan1307/lander.