Beyond Night Visibility: Adaptive Multi-Scale Fusion of Infrared and
  Visible Images
Shufan Pei Junhong Lin Wenxi Liu Tiesong Zhao Chia-Wen Lin
In addition to low light, night images suffer degradation from light effects
(e.g., glare, floodlight, etc). However, existing nighttime visibility
enhancement methods generally focus on low-light regions, which neglects, or
even amplifies the light effects. To address this issue, we propose an Adaptive
Multi-scale Fusion network (AMFusion) with infrared and visible images, which
designs fusion rules according to different illumination regions. First, we
separately fuse spatial and semantic features from infrared and visible images,
where the former are used for the adjustment of light distribution and the
latter are used for the improvement of detection accuracy. Thereby, we obtain
an image free of low light and light effects, which improves the performance of
nighttime object detection. Second, we utilize detection features extracted by
a pre-trained backbone that guide the fusion of semantic features. Hereby, we
design a Detection-guided Semantic Fusion Module (DSFM) to bridge the domain
gap between detection and semantic features. Third, we propose a new
illumination loss to constrain fusion image with normal light intensity.
Experimental results demonstrate the superiority of AMFusion with better visual
quality and detection accuracy. The source code will be released after the peer
review process.