Byzantine-Robust Clustered Federated Learning
Zhixu Tao Kun Yang Sanjeev R. Kulkarni
This paper focuses on the problem of adversarial attacks from Byzantine
machines in a Federated Learning setting where non-Byzantine machines can be
partitioned into disjoint clusters. In this setting, non-Byzantine machines in
the same cluster have the same underlying data distribution, and different
clusters of non-Byzantine machines have different learning tasks. Byzantine
machines can adversarially attack any cluster and disturb the training process
on clusters they attack. In the presence of Byzantine machines, the goal of our
work is to identify cluster membership of non-Byzantine machines and optimize
the models learned by each cluster. We adopt the Iterative Federated Clustering
Algorithm (IFCA) framework of Ghosh et al. (2020) to alternatively estimate
cluster membership and optimize models. In order to make this framework robust
against adversarial attacks from Byzantine machines, we use coordinate-wise
trimmed mean and coordinate-wise median aggregation methods used by Yin et al.
(2018). Specifically, we propose a new Byzantine-Robust Iterative Federated
Clustering Algorithm to improve on the results in Ghosh et al. (2019). We prove
a convergence rate for this algorithm for strongly convex loss functions. We
compare our convergence rate with the convergence rate of an existing
algorithm, and we demonstrate the performance of our algorithm on simulated
data.