An Interactively Reinforced Paradigm for Joint Infrared-Visible Image
  Fusion and Saliency Object Detection
Di Wang Jinyuan Liu Risheng Liu Xin Fan
This research focuses on the discovery and localization of hidden objects in
the wild and serves unmanned systems. Through empirical analysis, infrared and
visible image fusion (IVIF) enables hard-to-find objects apparent, whereas
multimodal salient object detection (SOD) accurately delineates the precise
spatial location of objects within the picture. Their common characteristic of
seeking complementary cues from different source images motivates us to explore
the collaborative relationship between Fusion and Salient object detection
tasks on infrared and visible images via an Interactively Reinforced multi-task
paradigm for the first time, termed IRFS. To the seamless bridge of multimodal
image fusion and SOD tasks, we specifically develop a Feature Screening-based
Fusion subnetwork (FSFNet) to screen out interfering features from source
images, thereby preserving saliency-related features. After generating the
fused image through FSFNet, it is then fed into the subsequent Fusion-Guided
Cross-Complementary SOD subnetwork (FC$^2$Net) as the third modality to drive
the precise prediction of the saliency map by leveraging the complementary
information derived from the fused image. In addition, we develop an
interactive loop learning strategy to achieve the mutual reinforcement of IVIF
and SOD tasks with a shorter training period and fewer network parameters.
Comprehensive experiment results demonstrate that the seamless bridge of IVIF
and SOD mutually enhances their performance, and highlights their superiority.