Temporal Adaptive RGBT Tracking with Modality Prompt
Hongyu Wang Xiaotao Liu Yifan Li Meng Sun Dian Yuan Jing Liu
RGBT tracking has been widely used in various fields such as robotics,
surveillance processing, and autonomous driving. Existing RGBT trackers fully
explore the spatial information between the template and the search region and
locate the target based on the appearance matching results. However, these RGBT
trackers have very limited exploitation of temporal information, either
ignoring temporal information or exploiting it through online sampling and
training. The former struggles to cope with the object state changes, while the
latter neglects the correlation between spatial and temporal information. To
alleviate these limitations, we propose a novel Temporal Adaptive RGBT Tracking
framework, named as TATrack. TATrack has a spatio-temporal two-stream structure
and captures temporal information by an online updated template, where the
two-stream structure refers to the multi-modal feature extraction and
cross-modal interaction for the initial template and the online update template
respectively. TATrack contributes to comprehensively exploit spatio-temporal
information and multi-modal information for target localization. In addition,
we design a spatio-temporal interaction (STI) mechanism that bridges two
branches and enables cross-modal interaction to span longer time scales.
Extensive experiments on three popular RGBT tracking benchmarks show that our
method achieves state-of-the-art performance, while running at real-time speed.