Reinforcement Learning as a Catalyst for Robust and Fair Federated
  Learning: Deciphering the Dynamics of Client Contributions
Jialuo He Wei Chen Xiaojin Zhang
Recent advancements in federated learning (FL) have produced models that
retain user privacy by training across multiple decentralized devices or
systems holding local data samples. However, these strategies often neglect the
inherent challenges of statistical heterogeneity and vulnerability to
adversarial attacks, which can degrade model robustness and fairness.
Personalized FL strategies offer some respite by adjusting models to fit
individual client profiles, yet they tend to neglect server-side aggregation
vulnerabilities. To address these issues, we propose Reinforcement Federated
Learning (RFL), a novel framework that leverages deep reinforcement learning to
adaptively optimize client contribution during aggregation, thereby enhancing
both model robustness against malicious clients and fairness across
participants under non-identically distributed settings. To achieve this goal,
we propose a meticulous approach involving a Deep Deterministic Policy
Gradient-based algorithm for continuous control of aggregation weights, an
innovative client selection method based on model parameter distances, and a
reward mechanism guided by validation set performance. Empirically, extensive
experiments demonstrate that, in terms of robustness, RFL outperforms the
state-of-the-art methods, while maintaining comparable levels of fairness,
offering a promising solution to build resilient and fair federated systems.