Federated Reinforcement Learning for Electric Vehicles Charging Control
  on Distribution Networks
Junkai Qian Yuning Jiang Xin Liu Qing Wang Ting Wang Yuanming Shi Wei Chen
With the growing popularity of electric vehicles (EVs), maintaining power
grid stability has become a significant challenge. To address this issue, EV
charging control strategies have been developed to manage the switch between
vehicle-to-grid (V2G) and grid-to-vehicle (G2V) modes for EVs. In this context,
multi-agent deep reinforcement learning (MADRL) has proven its effectiveness in
EV charging control. However, existing MADRL-based approaches fail to consider
the natural power flow of EV charging/discharging in the distribution network
and ignore driver privacy. To deal with these problems, this paper proposes a
novel approach that combines multi-EV charging/discharging with a radial
distribution network (RDN) operating under optimal power flow (OPF) to
distribute power flow in real time. A mathematical model is developed to
describe the RDN load. The EV charging control problem is formulated as a
Markov Decision Process (MDP) to find an optimal charging control strategy that
balances V2G profits, RDN load, and driver anxiety. To effectively learn the
optimal EV charging control strategy, a federated deep reinforcement learning
algorithm named FedSAC is further proposed. Comprehensive simulation results
demonstrate the effectiveness and superiority of our proposed algorithm in
terms of the diversity of the charging control strategy, the power fluctuations
on RDN, the convergence efficiency, and the generalization ability.