MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D
  Object Detection
Hou-I Liu Christine Wu Jen-Hao Cheng Wenhao Chai Shian-Yun Wang Gaowen Liu Jenq-Neng Hwang Hong-Han Shuai Wen-Huang Cheng
Monocular 3D object detection (Mono3D) is an indispensable research topic in
autonomous driving, thanks to the cost-effective monocular camera sensors and
its wide range of applications. Since the image perspective has depth
ambiguity, the challenges of Mono3D lie in understanding 3D scene geometry and
reconstructing 3D object information from a single image. Previous methods
attempted to transfer 3D information directly from the LiDAR-based teacher to
the camera-based student. However, a considerable gap in feature representation
makes direct cross-modal distillation inefficient, resulting in a significant
performance deterioration between the LiDAR-based teacher and the camera-based
student. To address this issue, we propose the Teaching Assistant Knowledge
Distillation (MonoTAKD) to break down the learning objective by integrating
intra-modal distillation with cross-modal residual distillation. In particular,
we employ a strong camera-based teaching assistant model to distill powerful
visual knowledge effectively through intra-modal distillation. Subsequently, we
introduce the cross-modal residual distillation to transfer the 3D spatial
cues. By acquiring both visual knowledge and 3D spatial cues, the predictions
of our approach are rigorously evaluated on the KITTI 3D object detection
benchmark and achieve state-of-the-art performance in Mono3D.