Hybrid Reinforcement Learning for Optimizing Pump Sustainability in
  Real-World Water Distribution Networks
Harsh Patel Yuan Zhou Alexander P Lamb Shu Wang Jieliang Luo
This article addresses the pump-scheduling optimization problem to enhance
real-time control of real-world water distribution networks (WDNs). Our primary
objectives are to adhere to physical operational constraints while reducing
energy consumption and operational costs. Traditional optimization techniques,
such as evolution-based and genetic algorithms, often fall short due to their
lack of convergence guarantees. Conversely, reinforcement learning (RL) stands
out for its adaptability to uncertainties and reduced inference time, enabling
real-time responsiveness. However, the effective implementation of RL is
contingent on building accurate simulation models for WDNs, and prior
applications have been limited by errors in simulation training data. These
errors can potentially cause the RL agent to learn misleading patterns and
actions and recommend suboptimal operational strategies. To overcome these
challenges, we present an improved "hybrid RL" methodology. This method
integrates the benefits of RL while anchoring it in historical data, which
serves as a baseline to incrementally introduce optimal control
recommendations. By leveraging operational data as a foundation for the agent's
actions, we enhance the explainability of the agent's actions, foster more
robust recommendations, and minimize error. Our findings demonstrate that the
hybrid RL agent can significantly improve sustainability, operational
efficiency, and dynamically adapt to emerging scenarios in real-world WDNs.