FedCCL: Federated Dual-Clustered Feature Contrast Under Domain
  Heterogeneity
Yu Qiao Huy Q. Le Mengchun Zhang Apurba Adhikary Chaoning Zhang Choong Seon Hong
Federated learning (FL) facilitates a privacy-preserving neural network
training paradigm through collaboration between edge clients and a central
server. One significant challenge is that the distributed data is not
independently and identically distributed (non-IID), typically including both
intra-domain and inter-domain heterogeneity. However, recent research is
limited to simply using averaged signals as a form of regularization and only
focusing on one aspect of these non-IID challenges. Given these limitations,
this paper clarifies these two non-IID challenges and attempts to introduce
cluster representation to address them from both local and global perspectives.
Specifically, we propose a dual-clustered feature contrast-based FL framework
with dual focuses. First, we employ clustering on the local representations of
each client, aiming to capture intra-class information based on these local
clusters at a high level of granularity. Then, we facilitate cross-client
knowledge sharing by pulling the local representation closer to clusters shared
by clients with similar semantics while pushing them away from clusters with
dissimilar semantics. Second, since the sizes of local clusters belonging to
the same class may differ for each client, we further utilize clustering on the
global side and conduct averaging to create a consistent global signal for
guiding each local training in a contrastive manner. Experimental results on
multiple datasets demonstrate that our proposal achieves comparable or superior
performance gain under intra-domain and inter-domain heterogeneity.