PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant
  Semantic Segmentation
Zhu Liu Jinyuan Liu Benzhuang Zhang Long Ma Xin Fan Risheng Liu
Infrared and visible image fusion is a powerful technique that combines
complementary information from different modalities for downstream semantic
perception tasks. Existing learning-based methods show remarkable performance,
but are suffering from the inherent vulnerability of adversarial attacks,
causing a significant decrease in accuracy. In this work, a perception-aware
fusion framework is proposed to promote segmentation robustness in adversarial
scenes. We first conduct systematic analyses about the components of image
fusion, investigating the correlation with segmentation robustness under
adversarial perturbations. Based on these analyses, we propose a harmonized
architecture search with a decomposition-based structure to balance standard
accuracy and robustness. We also propose an adaptive learning strategy to
improve the parameter robustness of image fusion, which can learn effective
feature extraction under diverse adversarial perturbations. Thus, the goals of
image fusion (\textit{i.e.,} extracting complementary features from source
modalities and defending attack) can be realized from the perspectives of
architectural and learning strategies. Extensive experimental results
demonstrate that our scheme substantially enhances the robustness, with gains
of 15.3% mIOU of segmentation in the adversarial scene, compared with advanced
competitors. The source codes are available at
https://github.com/LiuZhu-CV/PAIF.