Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian
  Approach
Mahrokh Ghoddousi Boroujeni Andreas Krause Giancarlo Ferrari Trecate
Federated learning aims to infer a shared model from private and
decentralized data stored locally by multiple clients. Personalized federated
learning (PFL) goes one step further by adapting the global model to each
client, enhancing the model's fit for different clients. A significant level of
personalization is required for highly heterogeneous clients, but can be
challenging to achieve especially when they have small datasets. To address
this problem, we propose a PFL algorithm named PAC-PFL for learning
probabilistic models within a PAC-Bayesian framework that utilizes differential
privacy to handle data-dependent priors. Our algorithm collaboratively learns a
shared hyper-posterior and regards each client's posterior inference as the
personalization step. By establishing and minimizing a generalization bound on
the average true risk of clients, PAC-PFL effectively combats over-fitting.
PACPFL achieves accurate and well-calibrated predictions, supported by
experiments on a dataset of photovoltaic panel power generation, FEMNIST
dataset (Caldas et al., 2019), and Dirichlet-partitioned EMNIST dataset (Cohen
et al., 2017).