Advancing Personalized Federated Learning: Group Privacy, Fairness, and
  Beyond
Filippo Galli Kangsoo Jung Sayan Biswas Catuscia Palamidessi Tommaso Cucinotta
Federated learning (FL) is a framework for training machine learning models
in a distributed and collaborative manner. During training, a set of
participating clients process their data stored locally, sharing only the model
updates obtained by minimizing a cost function over their local inputs. FL was
proposed as a stepping-stone towards privacy-preserving machine learning, but
it has been shown vulnerable to issues such as leakage of private information,
lack of personalization of the model, and the possibility of having a trained
model that is fairer to some groups than to others. In this paper, we address
the triadic interaction among personalization, privacy guarantees, and fairness
attained by models trained within the FL framework. Differential privacy and
its variants have been studied and applied as cutting-edge standards for
providing formal privacy guarantees. However, clients in FL often hold very
diverse datasets representing heterogeneous communities, making it important to
protect their sensitive information while still ensuring that the trained model
upholds the aspect of fairness for the users. To attain this objective, a
method is put forth that introduces group privacy assurances through the
utilization of $d$-privacy (aka metric privacy). $d$-privacy represents a
localized form of differential privacy that relies on a metric-oriented
obfuscation approach to maintain the original data's topological distribution.
This method, besides enabling personalized model training in a federated
approach and providing formal privacy guarantees, possesses significantly
better group fairness measured under a variety of standard metrics than a
global model trained within a classical FL template. Theoretical justifications
for the applicability are provided, as well as experimental validation on
real-world datasets to illustrate the working of the proposed method.