Robustness Testing for Multi-Agent Reinforcement Learning: State
  Perturbations on Critical Agents
Ziyuan Zhou Guanjun Liu
Multi-Agent Reinforcement Learning (MARL) has been widely applied in many
fields such as smart traffic and unmanned aerial vehicles. However, most MARL
algorithms are vulnerable to adversarial perturbations on agent states.
Robustness testing for a trained model is an essential step for confirming the
trustworthiness of the model against unexpected perturbations. This work
proposes a novel Robustness Testing framework for MARL that attacks states of
Critical Agents (RTCA). The RTCA has two innovations: 1) a Differential
Evolution (DE) based method to select critical agents as victims and to advise
the worst-case joint actions on them; and 2) a team cooperation policy
evaluation method employed as the objective function for the optimization of
DE. Then, adversarial state perturbations of the critical agents are generated
based on the worst-case joint actions. This is the first robustness testing
framework with varying victim agents. RTCA demonstrates outstanding performance
in terms of the number of victim agents and destroying cooperation policies.