Federated Generalized Category Discovery
Nan Pu Zhun Zhong Xinyuan Ji Nicu Sebe
Generalized category discovery (GCD) aims at grouping unlabeled samples from
known and unknown classes, given labeled data of known classes. To meet the
recent decentralization trend in the community, we introduce a practical yet
challenging task, namely Federated GCD (Fed-GCD), where the training data are
distributively stored in local clients and cannot be shared among clients. The
goal of Fed-GCD is to train a generic GCD model by client collaboration under
the privacy-protected constraint. The Fed-GCD leads to two challenges: 1)
representation degradation caused by training each client model with fewer data
than centralized GCD learning, and 2) highly heterogeneous label spaces across
different clients. To this end, we propose a novel Associated Gaussian
Contrastive Learning (AGCL) framework based on learnable GMMs, which consists
of a Client Semantics Association (CSA) and a global-local GMM Contrastive
Learning (GCL). On the server, CSA aggregates the heterogeneous categories of
local-client GMMs to generate a global GMM containing more comprehensive
category knowledge. On each client, GCL builds class-level contrastive learning
with both local and global GMMs. The local GCL learns robust representation
with limited local data. The global GCL encourages the model to produce more
discriminative representation with the comprehensive category relationships
that may not exist in local data. We build a benchmark based on six visual
datasets to facilitate the study of Fed-GCD. Extensive experiments show that
our AGCL outperforms the FedAvg-based baseline on all datasets.