MotionTrack: End-to-End Transformer-based Multi-Object Tracing with
  LiDAR-Camera Fusion
Ce Zhang Chengjie Zhang Yiluan Guo Lingji Chen Michael Happold
Multiple Object Tracking (MOT) is crucial to autonomous vehicle perception.
End-to-end transformer-based algorithms, which detect and track objects
simultaneously, show great potential for the MOT task. However, most existing
methods focus on image-based tracking with a single object category. In this
paper, we propose an end-to-end transformer-based MOT algorithm (MotionTrack)
with multi-modality sensor inputs to track objects with multiple classes. Our
objective is to establish a transformer baseline for the MOT in an autonomous
driving environment. The proposed algorithm consists of a transformer-based
data association (DA) module and a transformer-based query enhancement module
to achieve MOT and Multiple Object Detection (MOD) simultaneously. The
MotionTrack and its variations achieve better results (AMOTA score at 0.55) on
the nuScenes dataset compared with other classical baseline models, such as the
AB3DMOT, the CenterTrack, and the probabilistic 3D Kalman filter. In addition,
we prove that a modified attention mechanism can be utilized for DA to
accomplish the MOT, and aggregate history features to enhance the MOD
performance.