Optimal Economic Gas Turbine Dispatch with Deep Reinforcement Learning
Manuel Sage Martin Staniszewski Yaoyao Fiona Zhao
Dispatching strategies for gas turbines (GTs) are changing in modern
electricity grids. A growing incorporation of intermittent renewable energy
requires GTs to operate more but shorter cycles and more frequently on partial
loads. Deep reinforcement learning (DRL) has recently emerged as a tool that
can cope with this development and dispatch GTs economically. The key
advantages of DRL are a model-free optimization and the ability to handle
uncertainties, such as those introduced by varying loads or renewable energy
production. In this study, three popular DRL algorithms are implemented for an
economic GT dispatch problem on a case study in Alberta, Canada. We highlight
the benefits of DRL by incorporating an existing thermodynamic software
provided by Siemens Energy into the environment model and by simulating
uncertainty via varying electricity prices, loads, and ambient conditions.
Among the tested algorithms and baseline methods, Deep Q-Networks (DQN)
obtained the highest rewards while Proximal Policy Optimization (PPO) was the
most sample efficient. We further propose and implement a method to assign GT
operation and maintenance cost dynamically based on operating hours and cycles.
Compared to existing methods, our approach better approximates the true cost of
modern GT dispatch and hence leads to more realistic policies.