Personalized Federated Learning via Amortized Bayesian Meta-Learning
Shiyu Liu Shaogao Lv Dun Zeng Zenglin Xu Hui Wang Yue Yu
Federated learning is a decentralized and privacy-preserving technique that
enables multiple clients to collaborate with a server to learn a global model
without exposing their private data. However, the presence of statistical
heterogeneity among clients poses a challenge, as the global model may struggle
to perform well on each client's specific task. To address this issue, we
introduce a new perspective on personalized federated learning through
Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm
called \emph{FedABML}, which employs hierarchical variational inference across
clients. The global prior aims to capture representations of common intrinsic
structures from heterogeneous clients, which can then be transferred to their
respective tasks and aid in the generation of accurate client-specific
approximate posteriors through a few local updates. Our theoretical analysis
provides an upper bound on the average generalization error and guarantees the
generalization performance on unseen data. Finally, several empirical results
are implemented to demonstrate that \emph{FedABML} outperforms several
competitive baselines.