Prototype-based Cross-Modal Object Tracking
Lei Liu Chenglong Li Futian Wang Longfeng Shen Jin Tang
Cross-modal object tracking is an important research topic in the field of
information fusion, and it aims to address imaging limitations in challenging
scenarios by integrating switchable visible and near-infrared modalities.
However, existing tracking methods face some difficulties in adapting to
significant target appearance variations in the presence of modality switch.
For instance, model update based tracking methods struggle to maintain stable
tracking results during modality switching, leading to error accumulation and
model drift. Template based tracking methods solely rely on the template
information from first frame and/or last frame, which lacks sufficient
representation ability and poses challenges in handling significant target
appearance changes. To address this problem, we propose a prototype-based
cross-modal object tracker called ProtoTrack, which introduces a novel
prototype learning scheme to adapt to significant target appearance variations,
for cross-modal object tracking. In particular, we design a multi-modal
prototype to represent target information by multi-kind samples, including a
fixed sample from the first frame and two representative samples from different
modalities. Moreover, we develop a prototype generation algorithm based on two
new modules to ensure the prototype representative in different
challenges......