FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via
  Weight Shift Aggregation
Jungwon Seo Chunming Rong Minhoe Kim
Federated Learning (FL) offers a compelling method for training machine
learning models with a focus on preserving data privacy. The presence of system
heterogeneity and statistical heterogeneity, recognized challenges in FL,
arises from the diversity of client hardware, network, and dataset
distribution. This diversity can critically affect the training pace and the
performance of models. While many studies address either system or statistical
heterogeneity by introducing communication-efficient or stable convergence
algorithms, addressing these challenges in isolation often leads to compromises
due to unaddressed heterogeneity. In response, this paper introduces FedShift,
a novel algorithm designed to enhance both the training speed and the models'
accuracy in a dual heterogeneity scenario. Our solution can improve client
engagement through quantization and mitigate the adverse effects on performance
typically associated with quantization by employing a shifting technique. This
technique has proven to enhance accuracy by an average of 3.9% in diverse
heterogeneity environments.