Advancing Attack-Resilient Scheduling of Integrated Energy Systems with
  Demand Response via Deep Reinforcement Learning
Yang Li Wenjie Ma Yuanzheng Li Sen Li Zhe Chen
Optimally scheduling multi-energy flow is an effective method to utilize
renewable energy sources (RES) and improve the stability and economy of
integrated energy systems (IES). However, the stable demand-supply of IES faces
challenges from uncertainties that arise from RES and loads, as well as the
increasing impact of cyber-attacks with advanced information and communication
technologies adoption. To address these challenges, this paper proposes an
innovative model-free resilience scheduling method based on state-adversarial
deep reinforcement learning (DRL) for integrated demand response (IDR)-enabled
IES. The proposed method designs an IDR program to explore the interaction
ability of electricity-gas-heat flexible loads. Additionally, a
state-adversarial Markov decision process (SA-MDP) model characterizes the
energy scheduling problem of IES under cyber-attack. The state-adversarial soft
actor-critic (SA-SAC) algorithm is proposed to mitigate the impact of
cyber-attacks on the scheduling strategy. Simulation results demonstrate that
our method is capable of adequately addressing the uncertainties resulting from
RES and loads, mitigating the impact of cyber-attacks on the scheduling
strategy, and ensuring a stable demand supply for various energy sources.
Moreover, the proposed method demonstrates resilience against cyber-attacks.
Compared to the original soft actor-critic (SAC) algorithm, it achieves a 10\%
improvement in economic performance under cyber-attack scenarios.