MarineVRS: Marine Video Retrieval System with Explainability via
  Semantic Understanding
Tan-Sang Ha Hai Nguyen-Truong Tuan-Anh Vu Sai-Kit Yeung
Building a video retrieval system that is robust and reliable, especially for
the marine environment, is a challenging task due to several factors such as
dealing with massive amounts of dense and repetitive data, occlusion,
blurriness, low lighting conditions, and abstract queries. To address these
challenges, we present MarineVRS, a novel and flexible video retrieval system
designed explicitly for the marine domain. MarineVRS integrates
state-of-the-art methods for visual and linguistic object representation to
enable efficient and accurate search and analysis of vast volumes of underwater
video data. In addition, unlike the conventional video retrieval system, which
only permits users to index a collection of images or videos and search using a
free-form natural language sentence, our retrieval system includes an
additional Explainability module that outputs the segmentation masks of the
objects that the input query referred to. This feature allows users to identify
and isolate specific objects in the video footage, leading to more detailed
analysis and understanding of their behavior and movements. Finally, with its
adaptability, explainability, accuracy, and scalability, MarineVRS is a
powerful tool for marine researchers and scientists to efficiently and
accurately process vast amounts of data and gain deeper insights into the
behavior and movements of marine species.