Robust Electric Vehicle Balancing of Autonomous Mobility-On-Demand
  System: A Multi-Agent Reinforcement Learning Approach
Sihong He Shuo Han Fei Miao
Electric autonomous vehicles (EAVs) are getting attention in future
autonomous mobility-on-demand (AMoD) systems due to their economic and societal
benefits. However, EAVs' unique charging patterns (long charging time, high
charging frequency, unpredictable charging behaviors, etc.) make it challenging
to accurately predict the EAVs supply in E-AMoD systems. Furthermore, the
mobility demand's prediction uncertainty makes it an urgent and challenging
task to design an integrated vehicle balancing solution under supply and demand
uncertainties. Despite the success of reinforcement learning-based E-AMoD
balancing algorithms, state uncertainties under the EV supply or mobility
demand remain unexplored. In this work, we design a multi-agent reinforcement
learning (MARL)-based framework for EAVs balancing in E-AMoD systems, with
adversarial agents to model both the EAVs supply and mobility demand
uncertainties that may undermine the vehicle balancing solutions. We then
propose a robust E-AMoD Balancing MARL (REBAMA) algorithm to train a robust
EAVs balancing policy to balance both the supply-demand ratio and charging
utilization rate across the whole city. Experiments show that our proposed
robust method performs better compared with a non-robust MARL method that does
not consider state uncertainties; it improves the reward, charging utilization
fairness, and supply-demand fairness by 19.28%, 28.18%, and 3.97%,
respectively. Compared with a robust optimization-based method, the proposed
MARL algorithm can improve the reward, charging utilization fairness, and
supply-demand fairness by 8.21%, 8.29%, and 9.42%, respectively.