Siamese-DETR for Generic Multi-Object Tracking
Qiankun Liu Yichen Li Yuqi Jiang Ying Fu
The ability to detect and track the dynamic objects in different scenes is
fundamental to real-world applications, e.g., autonomous driving and robot
navigation. However, traditional Multi-Object Tracking (MOT) is limited to
tracking objects belonging to the pre-defined closed-set categories. Recently,
Open-Vocabulary MOT (OVMOT) and Generic MOT (GMOT) are proposed to track
interested objects beyond pre-defined categories with the given text prompt and
template image. However, the expensive well pre-trained (vision-)language model
and fine-grained category annotations are required to train OVMOT models. In
this paper, we focus on GMOT and propose a simple but effective method,
Siamese-DETR, for GMOT. Only the commonly used detection datasets (e.g., COCO)
are required for training. Different from existing GMOT methods, which train a
Single Object Tracking (SOT) based detector to detect interested objects and
then apply a data association based MOT tracker to get the trajectories, we
leverage the inherent object queries in DETR variants. Specifically: 1) The
multi-scale object queries are designed based on the given template image,
which are effective for detecting different scales of objects with the same
category as the template image; 2) A dynamic matching training strategy is
introduced to train Siamese-DETR on commonly used detection datasets, which
takes full advantage of provided annotations; 3) The online tracking pipeline
is simplified through a tracking-by-query manner by incorporating the tracked
boxes in previous frame as additional query boxes. The complex data association
is replaced with the much simpler Non-Maximum Suppression (NMS). Extensive
experimental results show that Siamese-DETR surpasses existing MOT methods on
GMOT-40 dataset by a large margin.