Personalized Federated Learning with Feature Alignment and Classifier
  Collaboration
Jian Xu Xinyi Tong Shao-Lun Huang
Data heterogeneity is one of the most challenging issues in federated
learning, which motivates a variety of approaches to learn personalized models
for participating clients. One such approach in deep neural networks based
tasks is employing a shared feature representation and learning a customized
classifier head for each client. However, previous works do not utilize the
global knowledge during local representation learning and also neglect the
fine-grained collaboration between local classifier heads, which limit the
model generalization ability. In this work, we conduct explicit local-global
feature alignment by leveraging global semantic knowledge for learning a better
representation. Moreover, we quantify the benefit of classifier combination for
each client as a function of the combining weights and derive an optimization
problem for estimating optimal weights. Finally, extensive evaluation results
on benchmark datasets with various heterogeneous data scenarios demonstrate the
effectiveness of our proposed method. Code is available at
https://github.com/JianXu95/FedPAC