An Empirical Study on Google Research Football Multi-agent Scenarios
Yan Song He Jiang Zheng Tian Haifeng Zhang Yingping Zhang Jiangcheng Zhu Zonghong Dai Weinan Zhang Jun Wang
Few multi-agent reinforcement learning (MARL) research on Google Research
Football (GRF) focus on the 11v11 multi-agent full-game scenario and to the
best of our knowledge, no open benchmark on this scenario has been released to
the public. In this work, we fill the gap by providing a population-based MARL
training pipeline and hyperparameter settings on multi-agent football scenario
that outperforms the bot with difficulty 1.0 from scratch within 2 million
steps. Our experiments serve as a reference for the expected performance of
Independent Proximal Policy Optimization (IPPO), a state-of-the-art multi-agent
reinforcement learning algorithm where each agent tries to maximize its own
policy independently across various training configurations. Meanwhile, we
open-source our training framework Light-MALib which extends the MALib codebase
by distributed and asynchronized implementation with additional analytical
tools for football games. Finally, we provide guidance for building strong
football AI with population-based training and release diverse pretrained
policies for benchmarking. The goal is to provide the community with a head
start for whoever experiment their works on GRF and a simple-to-use
population-based training framework for further improving their agents through
self-play. The implementation is available at
https://github.com/Shanghai-Digital-Brain-Laboratory/DB-Football.