Synchronize Feature Extracting and Matching: A Single Branch Framework
  for 3D Object Tracking
Teli Ma Mengmeng Wang Jimin Xiao Huifeng Wu Yong Liu
Siamese network has been a de facto benchmark framework for 3D LiDAR object
tracking with a shared-parametric encoder extracting features from template and
search region, respectively. This paradigm relies heavily on an additional
matching network to model the cross-correlation/similarity of the template and
search region. In this paper, we forsake the conventional Siamese paradigm and
propose a novel single-branch framework, SyncTrack, synchronizing the feature
extracting and matching to avoid forwarding encoder twice for template and
search region as well as introducing extra parameters of matching network. The
synchronization mechanism is based on the dynamic affinity of the Transformer,
and an in-depth analysis of the relevance is provided theoretically. Moreover,
based on the synchronization, we introduce a novel Attentive Points-Sampling
strategy into the Transformer layers (APST), replacing the random/Farthest
Points Sampling (FPS) method with sampling under the supervision of attentive
relations between the template and search region. It implies connecting
point-wise sampling with the feature learning, beneficial to aggregating more
distinctive and geometric features for tracking with sparse points. Extensive
experiments on two benchmark datasets (KITTI and NuScenes) show that SyncTrack
achieves state-of-the-art performance in real-time tracking.