3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking
Shuxiao Ding Eike Rehder Lukas Schneider Marius Cordts Juergen Gall
Tracking 3D objects accurately and consistently is crucial for autonomous
vehicles, enabling more reliable downstream tasks such as trajectory prediction
and motion planning. Based on the substantial progress in object detection in
recent years, the tracking-by-detection paradigm has become a popular choice
due to its simplicity and efficiency. State-of-the-art 3D multi-object tracking
(MOT) approaches typically rely on non-learned model-based algorithms such as
Kalman Filter but require many manually tuned parameters. On the other hand,
learning-based approaches face the problem of adapting the training to the
online setting, leading to inevitable distribution mismatch between training
and inference as well as suboptimal performance. In this work, we propose
3DMOTFormer, a learned geometry-based 3D MOT framework building upon the
transformer architecture. We use an Edge-Augmented Graph Transformer to reason
on the track-detection bipartite graph frame-by-frame and conduct data
association via edge classification. To reduce the distribution mismatch
between training and inference, we propose a novel online training strategy
with an autoregressive and recurrent forward pass as well as sequential batch
optimization. Using CenterPoint detections, our approach achieves 71.2% and
68.2% AMOTA on the nuScenes validation and test split, respectively. In
addition, a trained 3DMOTFormer model generalizes well across different object
detectors. Code is available at: https://github.com/dsx0511/3DMOTFormer.