QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D
  Object Detection
Yifan Zhang Zhen Dong Huanrui Yang Ming Lu Cheng-Ching Tseng Yuan Du Kurt Keutzer Li Du Shanghang Zhang
Multi-view 3D detection based on BEV (bird-eye-view) has recently achieved
significant improvements. However, the huge memory consumption of
state-of-the-art models makes it hard to deploy them on vehicles, and the
non-trivial latency will affect the real-time perception of streaming
applications. Despite the wide application of quantization to lighten models,
we show in our paper that directly applying quantization in BEV tasks will 1)
make the training unstable, and 2) lead to intolerable performance degradation.
To solve these issues, our method QD-BEV enables a novel view-guided
distillation (VGD) objective, which can stabilize the quantization-aware
training (QAT) while enhancing the model performance by leveraging both image
features and BEV features. Our experiments show that QD-BEV achieves similar or
even better accuracy than previous methods with significant efficiency gains.
On the nuScenes datasets, the 4-bit weight and 6-bit activation quantized
QD-BEV-Tiny model achieves 37.2% NDS with only 15.8 MB model size,
outperforming BevFormer-Tiny by 1.8% with an 8x model compression. On the Small
and Base variants, QD-BEV models also perform superbly and achieve 47.9% NDS
(28.2 MB) and 50.9% NDS (32.9 MB), respectively.