Deep Learning Based Simulators for the Phosphorus Removal Process
  Control in Wastewater Treatment via Deep Reinforcement Learning Algorithms
Esmaeel Mohammadi Mikkel Stokholm-Bjerregaard Aviaja Anna Hansen Per Halkjær Nielsen Daniel Ortiz-Arroyo Petar Durdevic
Phosphorus removal is vital in wastewater treatment to reduce reliance on
limited resources. Deep reinforcement learning (DRL) is a machine learning
technique that can optimize complex and nonlinear systems, including the
processes in wastewater treatment plants, by learning control policies through
trial and error. However, applying DRL to chemical and biological processes is
challenging due to the need for accurate simulators. This study trained six
models to identify the phosphorus removal process and used them to create a
simulator for the DRL environment. Although the models achieved high accuracy
(>97%), uncertainty and incorrect prediction behavior limited their performance
as simulators over longer horizons. Compounding errors in the models'
predictions were identified as one of the causes of this problem. This approach
for improving process control involves creating simulation environments for DRL
algorithms, using data from supervisory control and data acquisition (SCADA)
systems with a sufficient historical horizon without complex system modeling or
parameter estimation.