Efficient-VRNet: An Exquisite Fusion Network for Riverway Panoptic
  Perception based on Asymmetric Fair Fusion of Vision and 4D mmWave Radar
Runwei Guan Shanliang Yao Xiaohui Zhu Ka Lok Man Yong Yue Jeremy Smith Eng Gee Lim Yutao Yue
Panoptic perception is essential to unmanned surface vehicles (USVs) for
autonomous navigation. The current panoptic perception scheme is mainly based
on vision only, that is, object detection and semantic segmentation are
performed simultaneously based on camera sensors. Nevertheless, the fusion of
camera and radar sensors is regarded as a promising method which could
substitute pure vision methods, but almost all works focus on object detection
only. Therefore, how to maximize and subtly fuse the features of vision and
radar to improve both detection and segmentation is a challenge. In this paper,
we focus on riverway panoptic perception based on USVs, which is a considerably
unexplored field compared with road panoptic perception. We propose
Efficient-VRNet, a model based on Contextual Clustering (CoC) and the
asymmetric fusion of vision and 4D mmWave radar, which treats both vision and
radar modalities fairly. Efficient-VRNet can simultaneously perform detection
and segmentation of riverway objects and drivable area segmentation.
Furthermore, we adopt an uncertainty-based panoptic perception training
strategy to train Efficient-VRNet. In the experiments, our Efficient-VRNet
achieves better performances on our collected dataset than other uni-modal
models, especially in adverse weather and environment with poor lighting
conditions. Our code and models are available at
\url{https://github.com/GuanRunwei/Efficient-VRNet}.