LightZero: A Unified Benchmark for Monte Carlo Tree Search in General
  Sequential Decision Scenarios
Yazhe Niu Yuan Pu Zhenjie Yang Xueyan Li Tong Zhou Jiyuan Ren Shuai Hu Hongsheng Li Yu Liu
Building agents based on tree-search planning capabilities with learned
models has achieved remarkable success in classic decision-making problems,
such as Go and Atari. However, it has been deemed challenging or even
infeasible to extend Monte Carlo Tree Search (MCTS) based algorithms to diverse
real-world applications, especially when these environments involve complex
action spaces and significant simulation costs, or inherent stochasticity. In
this work, we introduce LightZero, the first unified benchmark for deploying
MCTS/MuZero in general sequential decision scenarios. Specificially, we
summarize the most critical challenges in designing a general MCTS-style
decision-making solver, then decompose the tightly-coupled algorithm and system
design of tree-search RL methods into distinct sub-modules. By incorporating
more appropriate exploration and optimization strategies, we can significantly
enhance these sub-modules and construct powerful LightZero agents to tackle
tasks across a wide range of domains, such as board games, Atari, MuJoCo,
MiniGrid and GoBigger. Detailed benchmark results reveal the significant
potential of such methods in building scalable and efficient decision
intelligence. The code is available as part of OpenDILab at
https://github.com/opendilab/LightZero.