FedCode: Communication-Efficient Federated Learning via Transferring
  Codebooks
Saeed Khalilian Vasileios Tsouvalas Tanir Ozcelebi Nirvana Meratnia
Federated Learning (FL) is a distributed machine learning paradigm that
enables learning models from decentralized local data. While FL offers
appealing properties for clients' data privacy, it imposes high communication
burdens for exchanging model weights between a server and the clients. Existing
approaches rely on model compression techniques, such as pruning and weight
clustering to tackle this. However, transmitting the entire set of weight
updates at each federated round, even in a compressed format, limits the
potential for a substantial reduction in communication volume. We propose
FedCode where clients transmit only codebooks, i.e., the cluster centers of
updated model weight values. To ensure a smooth learning curve and proper
calibration of clusters between the server and the clients, FedCode
periodically transfers model weights after multiple rounds of solely
communicating codebooks. This results in a significant reduction in
communication volume between clients and the server in both directions, without
imposing significant computational overhead on the clients or leading to major
performance degradation of the models. We evaluate the effectiveness of FedCode
using various publicly available datasets with ResNet-20 and MobileNet backbone
model architectures. Our evaluations demonstrate a 12.2-fold data transmission
reduction on average while maintaining a comparable model performance with an
average accuracy loss of 1.3% compared to FedAvg. Further validation of FedCode
performance under non-IID data distributions showcased an average accuracy loss
of 2.0% compared to FedAvg while achieving approximately a 12.7-fold data
transmission reduction.