Interpretable Deep Reinforcement Learning for Optimizing Heterogeneous
  Energy Storage Systems
Luolin Xiong Yang Tang Chensheng Liu Shuai Mao Ke Meng Zhaoyang Dong Feng Qian
Energy storage systems (ESS) are pivotal component in the energy market,
serving as both energy suppliers and consumers. ESS operators can reap benefits
from energy arbitrage by optimizing operations of storage equipment. To further
enhance ESS flexibility within the energy market and improve renewable energy
utilization, a heterogeneous photovoltaic-ESS (PV-ESS) is proposed, which
leverages the unique characteristics of battery energy storage (BES) and
hydrogen energy storage (HES). For scheduling tasks of the heterogeneous
PV-ESS, cost description plays a crucial role in guiding operator's strategies
to maximize benefits. We develop a comprehensive cost function that takes into
account degradation, capital, and operation/maintenance costs to reflect
real-world scenarios. Moreover, while numerous methods excel in optimizing ESS
energy arbitrage, they often rely on black-box models with opaque
decision-making processes, limiting practical applicability. To overcome this
limitation and enable transparent scheduling strategies, a prototype-based
policy network with inherent interpretability is introduced. This network
employs human-designed prototypes to guide decision-making by comparing
similarities between prototypical situations and encountered situations, which
allows for naturally explained scheduling strategies. Comparative results
across four distinct cases underscore the effectiveness and practicality of our
proposed pre-hoc interpretable optimization method when contrasted with
black-box models.