Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth
  and Data Heterogeneity
Yiyue Chen Haris Vikalo Chianing Wang
Motivated by high resource costs of centralized machine learning schemes as
well as data privacy concerns, federated learning (FL) emerged as an efficient
alternative that relies on aggregating locally trained models rather than
collecting clients' potentially private data. In practice, available resources
and data distributions vary from one client to another, creating an inherent
system heterogeneity that leads to deterioration of the performance of
conventional FL algorithms. In this work, we present a federated
quantization-based self-supervised learning scheme (Fed-QSSL) designed to
address heterogeneity in FL systems. At clients' side, to tackle data
heterogeneity we leverage distributed self-supervised learning while utilizing
low-bit quantization to satisfy constraints imposed by local infrastructure and
limited communication resources. At server's side, Fed-QSSL deploys
de-quantization, weighted aggregation and re-quantization, ultimately creating
models personalized to both data distribution as well as specific
infrastructure of each client's device. We validated the proposed algorithm on
real world datasets, demonstrating its efficacy, and theoretically analyzed
impact of low-bit training on the convergence and robustness of the learned
models.