Computer Vision for Carriers: PATRIOT
Ari Goodman Gurpreet Singh James Hing Ryan O'Shea
Deck tracking performed on carriers currently involves a team of sailors
manually identifying aircraft and updating a digital user interface called the
Ouija Board. Improvements to the deck tracking process would result in
increased Sortie Generation Rates, and therefore applying automation is seen as
a critical method to improve deck tracking. However, the requirements on a
carrier ship do not allow for the installation of hardware-based location
sensing technologies like Global Positioning System (GPS) sensors. PATRIOT
(Panoramic Asset Tracking of Real-Time Information for the Ouija Tabletop) is a
research effort and proposed solution to performing deck tracking with passive
sensing and without the need for GPS sensors. PATRIOT is a prototype system
which takes existing camera feeds, calculates aircraft poses, and updates a
virtual Ouija board interface with the current status of the assets. PATRIOT
would allow for faster, more accurate, and less laborious asset tracking for
aircraft, people, and support equipment. PATRIOT is anticipated to benefit the
warfighter by reducing cognitive workload, reducing manning requirements,
collecting data to improve logistics, and enabling an automation gateway for
future efforts to improve efficiency and safety. The authors have developed and
tested algorithms to perform pose estimations of assets in real-time including
OpenPifPaf, High-Resolution Network (HRNet), HigherHRNet (HHRNet), Faster
R-CNN, and in-house developed encoder-decoder network. The software was tested
with synthetic and real-world data and was able to accurately extract the pose
of assets. Fusion, tracking, and real-world generality are planned to be
improved to ensure a successful transition to the fleet.