Scalable Volt-VAR Optimization using RLlib-IMPALA Framework: A
  Reinforcement Learning Approach
Alaa Selim Yanzhu Ye Junbo Zhao Bo Yang
In the rapidly evolving domain of electrical power systems, the Volt-VAR
optimization (VVO) is increasingly critical, especially with the burgeoning
integration of renewable energy sources. Traditional approaches to
learning-based VVO in expansive and dynamically changing power systems are
often hindered by computational complexities. To address this challenge, our
research presents a novel framework that harnesses the potential of Deep
Reinforcement Learning (DRL), specifically utilizing the Importance Weighted
Actor-Learner Architecture (IMPALA) algorithm, executed on the RAY platform.
This framework, built upon RLlib-an industry-standard in Reinforcement
Learning-ingeniously capitalizes on the distributed computing capabilities and
advanced hyperparameter tuning offered by RAY. This design significantly
expedites the exploration and exploitation phases in the VVO solution space.
Our empirical results demonstrate that our approach not only surpasses existing
DRL methods in achieving superior reward outcomes but also manifests a
remarkable tenfold reduction in computational requirements. The integration of
our DRL agent with the RAY platform facilitates the creation of RLlib-IMPALA, a
novel framework that efficiently uses RAY's resources to improve system
adaptability and control. RLlib-IMPALA leverages RAY's toolkit to enhance
analytical capabilities and significantly speeds up training to become more
than 10 times faster than other state-of-the-art DRL methods.