Attentive Convolutional Deep Reinforcement Learning for Optimizing
  Solar-Storage Systems in Real-Time Electricity Markets
Jinhao Li Changlong Wang Hao Wang
This paper studies the synergy of solar-battery energy storage system (BESS)
and develops a viable strategy for the BESS to unlock its economic potential by
serving as a backup to reduce solar curtailments while also participating in
the electricity market. We model the real-time bidding of the solar-battery
system as two Markov decision processes for the solar farm and the BESS,
respectively. We develop a novel deep reinforcement learning (DRL) algorithm to
solve the problem by leveraging attention mechanism (AC) and multi-grained
feature convolution to process DRL input for better bidding decisions.
Simulation results demonstrate that our AC-DRL outperforms two
optimization-based and one DRL-based benchmarks by generating 23%, 20%, and 11%
higher revenue, as well as improving curtailment responses. The excess solar
generation can effectively charge the BESS to bid in the market, significantly
reducing solar curtailments by 76% and creating synergy for the solar-battery
system to be more viable.