Neuromorphic Vision-based Motion Segmentation with Graph Transformer
  Neural Network
Yusra Alkendi Rana Azzam Sajid Javed Lakmal Seneviratne Yahya Zweiri
Moving object segmentation is critical to interpret scene dynamics for
robotic navigation systems in challenging environments. Neuromorphic vision
sensors are tailored for motion perception due to their asynchronous nature,
high temporal resolution, and reduced power consumption. However, their
unconventional output requires novel perception paradigms to leverage their
spatially sparse and temporally dense nature. In this work, we propose a novel
event-based motion segmentation algorithm using a Graph Transformer Neural
Network, dubbed GTNN. Our proposed algorithm processes event streams as 3D
graphs by a series of nonlinear transformations to unveil local and global
spatiotemporal correlations between events. Based on these correlations, events
belonging to moving objects are segmented from the background without prior
knowledge of the dynamic scene geometry. The algorithm is trained on publicly
available datasets including MOD, EV-IMO, and \textcolor{black}{EV-IMO2} using
the proposed training scheme to facilitate efficient training on extensive
datasets. Moreover, we introduce the Dynamic Object Mask-aware Event Labeling
(DOMEL) approach for generating approximate ground-truth labels for event-based
motion segmentation datasets. We use DOMEL to label our own recorded Event
dataset for Motion Segmentation (EMS-DOMEL), which we release to the public for
further research and benchmarking. Rigorous experiments are conducted on
several unseen publicly-available datasets where the results revealed that GTNN
outperforms state-of-the-art methods in the presence of dynamic background
variations, motion patterns, and multiple dynamic objects with varying sizes
and velocities. GTNN achieves significant performance gains with an average
increase of 9.4% and 4.5% in terms of motion segmentation accuracy (IoU%) and
detection rate (DR%), respectively.