Cross-Modal Object Tracking via Modality-Aware Fusion Network and A
  Large-Scale Dataset
Lei Liu Mengya Zhang Cheng Li Chenglong Li Jin Tang
Visual tracking often faces challenges such as invalid targets and decreased
performance in low-light conditions when relying solely on RGB image sequences.
While incorporating additional modalities like depth and infrared data has
proven effective, existing multi-modal imaging platforms are complex and lack
real-world applicability. In contrast, near-infrared (NIR) imaging, commonly
used in surveillance cameras, can switch between RGB and NIR based on light
intensity. However, tracking objects across these heterogeneous modalities
poses significant challenges, particularly due to the absence of modality
switch signals during tracking. To address these challenges, we propose an
adaptive cross-modal object tracking algorithm called Modality-Aware Fusion
Network (MAFNet). MAFNet efficiently integrates information from both RGB and
NIR modalities using an adaptive weighting mechanism, effectively bridging the
appearance gap and enabling a modality-aware target representation. It consists
of two key components: an adaptive weighting module and a modality-specific
representation module......