LabelFormer: Object Trajectory Refinement for Offboard Perception from
  LiDAR Point Clouds
Anqi Joyce Yang Sergio Casas Nikita Dvornik Sean Segal Yuwen Xiong Jordan Sir Kwang Hu Carter Fang Raquel Urtasun
A major bottleneck to scaling-up training of self-driving perception systems
are the human annotations required for supervision. A promising alternative is
to leverage "auto-labelling" offboard perception models that are trained to
automatically generate annotations from raw LiDAR point clouds at a fraction of
the cost. Auto-labels are most commonly generated via a two-stage approach --
first objects are detected and tracked over time, and then each object
trajectory is passed to a learned refinement model to improve accuracy. Since
existing refinement models are overly complex and lack advanced temporal
reasoning capabilities, in this work we propose LabelFormer, a simple,
efficient, and effective trajectory-level refinement approach. Our approach
first encodes each frame's observations separately, then exploits
self-attention to reason about the trajectory with full temporal context, and
finally decodes the refined object size and per-frame poses. Evaluation on both
urban and highway datasets demonstrates that LabelFormer outperforms existing
works by a large margin. Finally, we show that training on a dataset augmented
with auto-labels generated by our method leads to improved downstream detection
performance compared to existing methods. Please visit the project website for
details https://waabi.ai/labelformer