Personalized Federated Learning with Contextual Modulation and
  Meta-Learning
Anna Vettoruzzo Mohamed-Rafik Bouguelia Thorsteinn Rögnvaldsson
Federated learning has emerged as a promising approach for training machine
learning models on decentralized data sources while preserving data privacy.
However, challenges such as communication bottlenecks, heterogeneity of client
devices, and non-i.i.d. data distribution pose significant obstacles to
achieving optimal model performance. We propose a novel framework that combines
federated learning with meta-learning techniques to enhance both efficiency and
generalization capabilities. Our approach introduces a federated modulator that
learns contextual information from data batches and uses this knowledge to
generate modulation parameters. These parameters dynamically adjust the
activations of a base model, which operates using a MAML-based approach for
model personalization. Experimental results across diverse datasets highlight
the improvements in convergence speed and model performance compared to
existing federated learning approaches. These findings highlight the potential
of incorporating contextual information and meta-learning techniques into
federated learning, paving the way for advancements in distributed machine
learning paradigms.