Gradient Coreset for Federated Learning
Durga Sivasubramanian Lokesh Nagalapatti Rishabh Iyer Ganesh Ramakrishnan
Federated Learning (FL) is used to learn machine learning models with data
that is partitioned across multiple clients, including resource-constrained
edge devices. It is therefore important to devise solutions that are efficient
in terms of compute, communication, and energy consumption, while ensuring
compliance with the FL framework's privacy requirements. Conventional
approaches to these problems select a weighted subset of the training dataset,
known as coreset, and learn by fitting models on it. Such coreset selection
approaches are also known to be robust to data noise. However, these approaches
rely on the overall statistics of the training data and are not easily
extendable to the FL setup.
  In this paper, we propose an algorithm called Gradient based Coreset for
Robust and Efficient Federated Learning (GCFL) that selects a coreset at each
client, only every $K$ communication rounds and derives updates only from it,
assuming the availability of a small validation dataset at the server. We
demonstrate that our coreset selection technique is highly effective in
accounting for noise in clients' data. We conduct experiments using four
real-world datasets and show that GCFL is (1) more compute and energy efficient
than FL, (2) robust to various kinds of noise in both the feature space and
labels, (3) preserves the privacy of the validation dataset, and (4) introduces
a small communication overhead but achieves significant gains in performance,
particularly in cases when the clients' data is noisy.