TaskFlex Solver for Multi-Agent Pursuit via Automatic Curriculum
  Learning
Jiayu Chen Guosheng Li Chao Yu Xinyi Yang Botian Xu Huazhong Yang Yu Wang
This paper addresses the problem of multi-agent pursuit, where slow pursuers
cooperate to capture fast evaders in a confined environment with obstacles.
Existing heuristic algorithms often lack expressive coordination strategies and
are highly sensitive to task conditions, requiring extensive hyperparameter
tuning. In contrast, reinforcement learning (RL) has been applied to this
problem and is capable of obtaining cooperative pursuit strategies. However,
RL-based methods face challenges in training for complex scenarios due to the
vast amount of training data and limited adaptability to varying task
conditions, such as different scene sizes, varying numbers and speeds of
obstacles, and flexible speed ratios of the evader to the pursuer. In this
work, we combine RL and curriculum learning to introduce a flexible solver for
multiagent pursuit problems, named TaskFlex Solver (TFS), which is capable of
solving multi-agent pursuit problems with diverse and dynamically changing task
conditions in both 2-dimensional and 3-dimensional scenarios. TFS utilizes a
curriculum learning method that constructs task distributions based on training
progress, enhancing training efficiency and final performance. Our algorithm
consists of two main components: the Task Evaluator, which evaluates task
success rates and selects tasks of moderate difficulty to maintain a curriculum
archive, and the Task Sampler, which constructs training distributions by
sampling tasks from the curriculum archive to maximize policy improvement.
Experiments show that TFS produces much stronger performance than baselines and
achieves close to 100% capture rates in both 2-dimensional and 3-dimensional
multi-agent pursuit problems with diverse and dynamically changing scenes. The
project website is at https://sites.google.com/view/tfs-2023.