Aeolus Ocean -- A simulation environment for the autonomous
  COLREG-compliant navigation of Unmanned Surface Vehicles using Deep
  Reinforcement Learning and Maritime Object Detection
Andrew Alexander Vekinis Stavros Perantonis
Heading towards navigational autonomy in unmanned surface vehicles (USVs) in
the maritime sector can fundamentally lead towards safer waters as well as
reduced operating costs, while also providing a range of exciting new
capabilities for oceanic research, exploration and monitoring. However,
achieving such a goal is challenging. USV control systems must, safely and
reliably, be able to adhere to the international regulations for preventing
collisions at sea (COLREGs) in encounters with other vessels as they navigate
to a given waypoint while being affected by realistic weather conditions,
either during the day or at night. To deal with the multitude of possible
scenarios, it is critical to have a virtual environment that is able to
replicate the realistic operating conditions USVs will encounter, before they
can be implemented in the real world. Such "digital twins" form the foundations
upon which Deep Reinforcement Learning (DRL) and Computer Vision (CV)
algorithms can be used to develop and guide USV control systems. In this paper
we describe the novel development of a COLREG-compliant DRL-based collision
avoidant navigational system with CV-based awareness in a realistic ocean
simulation environment. The performance of the trained autonomous Agents
resulting from this approach is evaluated in several successful navigations to
set waypoints in both open sea and coastal encounters with other vessels. A
binary executable version of the simulator with trained agents is available at
https://github.com/aavek/Aeolus-Ocean