Multiobjective Hydropower Reservoir Operation Optimization with
  Transformer-Based Deep Reinforcement Learning
Rixin Wu Ran Wang Jie Hao Qiang Wu Ping Wang
Due to shortage of water resources and increasing water demands, the joint
operation of multireservoir systems for balancing power generation, ecological
protection, and the residential water supply has become a critical issue in
hydropower management. However, the numerous constraints and nonlinearity of
multiple reservoirs make solving this problem time-consuming. To address this
challenge, a deep reinforcement learning approach that incorporates a
transformer framework is proposed. The multihead attention mechanism of the
encoder effectively extracts information from reservoirs and residential areas,
and the multireservoir attention network of the decoder generates suitable
operational decisions. The proposed method is applied to Lake Mead and Lake
Powell in the Colorado River Basin. The experimental results demonstrate that
the transformer-based deep reinforcement learning approach can produce
appropriate operational outcomes. Compared to a state-of-the-art method, the
operation strategies produced by the proposed approach generate 10.11% more
electricity, reduce the amended annual proportional flow deviation by 39.69%,
and increase water supply revenue by 4.10%. Consequently, the proposed approach
offers an effective method for the multiobjective operation of multihydropower
reservoir systems.