Multi-Agent Probabilistic Ensembles with Trajectory Sampling for
  Connected Autonomous Vehicles
Ruoqi Wen Jiahao Huang Rongpeng Li Guoru Ding Zhifeng Zhao
Autonomous Vehicles (AVs) have attracted significant attention in recent
years and Reinforcement Learning (RL) has shown remarkable performance in
improving the autonomy of vehicles. In that regard, the widely adopted
Model-Free RL (MFRL) promises to solve decision-making tasks in connected AVs
(CAVs), contingent on the readiness of a significant amount of data samples for
training. Nevertheless, it might be infeasible in practice and possibly lead to
learning instability. In contrast, Model-Based RL (MBRL) manifests itself in
sample-efficient learning, but the asymptotic performance of MBRL might lag
behind the state-of-the-art MFRL algorithms. Furthermore, most studies for CAVs
are limited to the decision-making of a single AV only, thus underscoring the
performance due to the absence of communications. In this study, we try to
address the decision-making problem of multiple CAVs with limited
communications and propose a decentralized Multi-Agent Probabilistic Ensembles
with Trajectory Sampling algorithm MA-PETS. In particular, in order to better
capture the uncertainty of the unknown environment, MA-PETS leverages
Probabilistic Ensemble (PE) neural networks to learn from communicated samples
among neighboring CAVs. Afterwards, MA-PETS capably develops Trajectory
Sampling (TS)-based model-predictive control for decision-making. On this
basis, we derive the multi-agent group regret bound affected by the number of
agents within the communication range and mathematically validate that
incorporating effective information exchange among agents into the multi-agent
learning scheme contributes to reducing the group regret bound in the worst
case. Finally, we empirically demonstrate the superiority of MA-PETS in terms
of the sample efficiency comparable to MFBL.