Correlation-Embedded Transformer Tracking: A Single-Branch Framework
Fei Xie Wankou Yang Chunyu Wang Lei Chu Yue Cao Chao Ma Wenjun Zeng
Developing robust and discriminative appearance models has been a
long-standing research challenge in visual object tracking. In the prevalent
Siamese-based paradigm, the features extracted by the Siamese-like networks are
often insufficient to model the tracked targets and distractor objects, thereby
hindering them from being robust and discriminative simultaneously. While most
Siamese trackers focus on designing robust correlation operations, we propose a
novel single-branch tracking framework inspired by the transformer. Unlike the
Siamese-like feature extraction, our tracker deeply embeds cross-image feature
correlation in multiple layers of the feature network. By extensively matching
the features of the two images through multiple layers, it can suppress
non-target features, resulting in target-aware feature extraction. The output
features can be directly used for predicting target locations without
additional correlation steps. Thus, we reformulate the two-branch Siamese
tracking as a conceptually simple, fully transformer-based Single-Branch
Tracking pipeline, dubbed SBT. After conducting an in-depth analysis of the SBT
baseline, we summarize many effective design principles and propose an improved
tracker dubbed SuperSBT. SuperSBT adopts a hierarchical architecture with a
local modeling layer to enhance shallow-level features. A unified relation
modeling is proposed to remove complex handcrafted layer pattern designs.
SuperSBT is further improved by masked image modeling pre-training, integrating
temporal modeling, and equipping with dedicated prediction heads. Thus,
SuperSBT outperforms the SBT baseline by 4.7%,3.0%, and 4.5% AUC scores in
LaSOT, TrackingNet, and GOT-10K. Notably, SuperSBT greatly raises the speed of
SBT from 37 FPS to 81 FPS. Extensive experiments show that our method achieves
superior results on eight VOT benchmarks.