A Deep Q-Learning based Smart Scheduling of EVs for Demand Response in
  Smart Grids
Viorica Rozina Chifu Tudor Cioara Cristina Bianca Pop Horia Rusu Ionut Anghel
Economic and policy factors are driving the continuous increase in the
adoption and usage of electrical vehicles (EVs). However, despite being a
cleaner alternative to combustion engine vehicles, EVs have negative impacts on
the lifespan of microgrid equipment and energy balance due to increased power
demand and the timing of their usage. In our view grid management should
leverage on EVs scheduling flexibility to support local network balancing
through active participation in demand response programs. In this paper, we
propose a model-free solution, leveraging Deep Q-Learning to schedule the
charging and discharging activities of EVs within a microgrid to align with a
target energy profile provided by the distribution system operator. We adapted
the Bellman Equation to assess the value of a state based on specific rewards
for EV scheduling actions and used a neural network to estimate Q-values for
available actions and the epsilon-greedy algorithm to balance exploitation and
exploration to meet the target energy profile. The results are promising
showing that the proposed solution can effectively schedule the EVs charging
and discharging actions to align with the target profile with a Person
coefficient of 0.99, handling effective EVs scheduling situations that involve
dynamicity given by the e-mobility features, relying only on data with no
knowledge of EVs and microgrid dynamics.