FedCore: Straggler-Free Federated Learning with Distributed Coresets
Hongpeng Guo Haotian Gu Xiaoyang Wang Bo Chen Eun Kyung Lee Tamar Eilam Deming Chen Klara Nahrstedt
Federated learning (FL) is a machine learning paradigm that allows multiple
clients to collaboratively train a shared model while keeping their data
on-premise. However, the straggler issue, due to slow clients, often hinders
the efficiency and scalability of FL. This paper presents FedCore, an algorithm
that innovatively tackles the straggler problem via the decentralized selection
of coresets, representative subsets of a dataset. Contrary to existing
centralized coreset methods, FedCore creates coresets directly on each client
in a distributed manner, ensuring privacy preservation in FL. FedCore
translates the coreset optimization problem into a more tractable k-medoids
clustering problem and operates distributedly on each client. Theoretical
analysis confirms FedCore's convergence, and practical evaluations demonstrate
an 8x reduction in FL training time, without compromising model accuracy. Our
extensive evaluations also show that FedCore generalizes well to existing FL
frameworks.