Multi-agent Deep Reinforcement Learning for Dynamic Pricing by
  Fast-charging Electric Vehicle Hubs in ccompetition
Diwas Paudel Tapas K. Das
Fast-charging hubs for electric vehicles will soon become part of the newly
built infrastructure for transportation electrification across the world. These
hubs are expected to host many DC fast-charging stations and will admit EVs
only for charging. Like the gasoline refueling stations, fast-charging hubs in
a neighborhood will dynamically vary their prices to compete for the same pool
of EV owners. These hubs will interact with the electric power network by
making purchase commitments for a significant part of their power needs in the
day-ahead (DA) electricity market and meeting the difference from the real-time
(RT) market. Hubs may have supplemental battery storage systems (BSS), which
they will use for arbitrage. In this paper, we develop a two-step data-driven
dynamic pricing methodology for hubs in price competition. We first obtain the
DA commitment by solving a stochastic DA commitment model. Thereafter we obtain
the hub pricing strategies by modeling the game as a competitive Markov
decision process (CMDP) and solving it using a multi-agent deep reinforcement
learning (MADRL) approach. We develop a numerical case study for a pricing game
between two charging hubs. We solve the case study with our methodology by
using combinations of two different DRL algorithms, DQN and SAC, and two
different neural networks (NN) architectures, a feed-forward (FF) neural
network, and a multi-head attention (MHA) neural network. We construct a
measure of collusion (index) using the hub profits. A value of zero for this
index indicates no collusion (perfect competition) and a value of one indicates
full collusion (monopolistic behavior). Our results show that the collusion
index varies approximately between 0.14 and 0.45 depending on the combinations
of the algorithms and the architectures chosen by the hubs.