Safety Aware Autonomous Path Planning Using Model Predictive
  Reinforcement Learning for Inland Waterways
Astrid Vanneste Simon Vanneste Olivier Vasseur Robin Janssens Mattias Billast Ali Anwar Kevin Mets Tom De Schepper Siegfried Mercelis Peter Hellinckx
In recent years, interest in autonomous shipping in urban waterways has
increased significantly due to the trend of keeping cars and trucks out of city
centers. Classical approaches such as Frenet frame based planning and potential
field navigation often require tuning of many configuration parameters and
sometimes even require a different configuration depending on the situation. In
this paper, we propose a novel path planning approach based on reinforcement
learning called Model Predictive Reinforcement Learning (MPRL). MPRL calculates
a series of waypoints for the vessel to follow. The environment is represented
as an occupancy grid map, allowing us to deal with any shape of waterway and
any number and shape of obstacles. We demonstrate our approach on two scenarios
and compare the resulting path with path planning using a Frenet frame and path
planning based on a proximal policy optimization (PPO) agent. Our results show
that MPRL outperforms both baselines in both test scenarios. The PPO based
approach was not able to reach the goal in either scenario while the Frenet
frame approach failed in the scenario consisting of a corner with obstacles.
MPRL was able to safely (collision free) navigate to the goal in both of the
test scenarios.