Global Convergence Guarantees for Federated Policy Gradient Methods with
  Adversaries
Swetha Ganesh Jiayu Chen Gugan Thoppe Vaneet Aggarwal
Federated Reinforcement Learning (FRL) allows multiple agents to
collaboratively build a decision making policy without sharing raw
trajectories. However, if a small fraction of these agents are adversarial, it
can lead to catastrophic results. We propose a policy gradient based approach
that is robust to adversarial agents which can send arbitrary values to the
server. Under this setting, our results form the first global convergence
guarantees with general parametrization. These results demonstrate resilience
with adversaries, while achieving sample complexity of order
$\tilde{\mathcal{O}}\left( \frac{1}{\epsilon^2} \left( \frac{1}{N-f} +
\frac{f^2}{(N-f)^2}\right)\right)$, where $N$ is the total number of agents and
$f$ is the number of adversarial agents.