EfficientMFD: Towards More Efficient Multimodal Synchronous Fusion
  Detection
Jiaqing Zhang Mingxiang Cao Xue Yang Weiying Xie Jie Lei Daixun Li Geng Yang Wenbo Huang Yunsong Li
Multimodal image fusion and object detection play a vital role in autonomous
driving. Current joint learning methods have made significant progress in the
multimodal fusion detection task combining the texture detail and objective
semantic information. However, the tedious training steps have limited its
applications to wider real-world industrial deployment. To address this
limitation, we propose a novel end-to-end multimodal fusion detection
algorithm, named EfficientMFD, to simplify models that exhibit decent
performance with only one training step. Synchronous joint optimization is
utilized in an end-to-end manner between two components, thus not being
affected by the local optimal solution of the individual task. Besides, a
comprehensive optimization is established in the gradient matrix between the
shared parameters for both tasks. It can converge to an optimal point with
fusion detection weights. We extensively test it on several public datasets,
demonstrating superior performance on not only visually appealing fusion but
also favorable detection performance (e.g., 6.6% mAP50:95) over other
state-of-the-art approaches.