Effectively Heterogeneous Federated Learning: A Pairing and Split
  Learning Based Approach
Jinglong Shen Xiucheng Wang Nan Cheng Longfei Ma Conghao Zhou Yuan Zhang
As a promising paradigm federated Learning (FL) is widely used in
privacy-preserving machine learning, which allows distributed devices to
collaboratively train a model while avoiding data transmission among clients.
Despite its immense potential, the FL suffers from bottlenecks in training
speed due to client heterogeneity, leading to escalated training latency and
straggling server aggregation. To deal with this challenge, a novel split
federated learning (SFL) framework that pairs clients with different
computational resources is proposed, where clients are paired based on
computing resources and communication rates among clients, meanwhile the neural
network model is split into two parts at the logical level, and each client
only computes the part assigned to it by using the SL to achieve forward
inference and backward training. Moreover, to effectively deal with the client
pairing problem, a heuristic greedy algorithm is proposed by reconstructing the
optimization of training latency as a graph edge selection problem. Simulation
results show the proposed method can significantly improve the FL training
speed and achieve high performance both in independent identical distribution
(IID) and Non-IID data distribution.