Scalable Semantic 3D Mapping of Coral Reefs with Deep Learning
Jonathan Sauder Guilhem Banc-Prandi Anders Meibom Devis Tuia
Coral reefs are among the most diverse ecosystems on our planet, and are
depended on by hundreds of millions of people. Unfortunately, most coral reefs
are existentially threatened by global climate change and local anthropogenic
pressures. To better understand the dynamics underlying deterioration of reefs,
monitoring at high spatial and temporal resolution is key. However,
conventional monitoring methods for quantifying coral cover and species
abundance are limited in scale due to the extensive manual labor required.
Although computer vision tools have been employed to aid in this process, in
particular SfM photogrammetry for 3D mapping and deep neural networks for image
segmentation, analysis of the data products creates a bottleneck, effectively
limiting their scalability. This paper presents a new paradigm for mapping
underwater environments from ego-motion video, unifying 3D mapping systems that
use machine learning to adapt to challenging conditions under water, combined
with a modern approach for semantic segmentation of images. The method is
exemplified on coral reefs in the northern Gulf of Aqaba, Red Sea,
demonstrating high-precision 3D semantic mapping at unprecedented scale with
significantly reduced required labor costs: a 100 m video transect acquired
within 5 minutes of diving with a cheap consumer-grade camera can be fully
automatically analyzed within 5 minutes. Our approach significantly scales up
coral reef monitoring by taking a leap towards fully automatic analysis of
video transects. The method democratizes coral reef transects by reducing the
labor, equipment, logistics, and computing cost. This can help to inform
conservation policies more efficiently. The underlying computational method of
learning-based Structure-from-Motion has broad implications for fast low-cost
mapping of underwater environments other than coral reefs.