AdaFed: Fair Federated Learning via Adaptive Common Descent Direction
Shayan Mohajer Hamidi En-Hui Yang
Federated learning (FL) is a promising technology via which some edge
devices/clients collaboratively train a machine learning model orchestrated by
a server. Learning an unfair model is known as a critical problem in federated
learning, where the trained model may unfairly advantage or disadvantage some
of the devices. To tackle this problem, in this work, we propose AdaFed. The
goal of AdaFed is to find an updating direction for the server along which (i)
all the clients' loss functions are decreasing; and (ii) more importantly, the
loss functions for the clients with larger values decrease with a higher rate.
AdaFed adaptively tunes this common direction based on the values of local
gradients and loss functions. We validate the effectiveness of AdaFed on a
suite of federated datasets, and demonstrate that AdaFed outperforms
state-of-the-art fair FL methods.