Improving a Proportional Integral Controller with Reinforcement Learning
  on a Throttle Valve Benchmark
Paul Daoudi Bojan Mavkov Bogdan Robu Christophe Prieur Emmanuel Witrant Merwan Barlier Ludovic Dos Santos
This paper presents a learning-based control strategy for non-linear throttle
valves with an asymmetric hysteresis, leading to a near-optimal controller
without requiring any prior knowledge about the environment. We start with a
carefully tuned Proportional Integrator (PI) controller and exploit the recent
advances in Reinforcement Learning (RL) with Guides to improve the closed-loop
behavior by learning from the additional interactions with the valve. We test
the proposed control method in various scenarios on three different valves, all
highlighting the benefits of combining both PI and RL frameworks to improve
control performance in non-linear stochastic systems. In all the experimental
test cases, the resulting agent has a better sample efficiency than traditional
RL agents and outperforms the PI controller.