Solution Simplex Clustering for Heterogeneous Federated Learning
Dennis Grinwald Philipp Wiesner Shinichi Nakajima
We tackle a major challenge in federated learning (FL) -- achieving good
performance under highly heterogeneous client distributions. The difficulty
partially arises from two seemingly contradictory goals: learning a common
model by aggregating the information from clients, and learning local
personalized models that should be adapted to each local distribution. In this
work, we propose Solution Simplex Clustered Federated Learning (SosicFL) for
dissolving such contradiction. Based on the recent ideas of learning solution
simplices, SosicFL assigns a subregion in a simplex to each client, and
performs FL to learn a common solution simplex. This allows the client models
to possess their characteristics within the degrees of freedom in the solution
simplex, and at the same time achieves the goal of learning a global common
model. Our experiments show that SosicFL improves the performance and
accelerates the training process for global and personalized FL with minimal
computational overhead.