Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud
  Analysis
Peipei Li Xing Cui Yibo Hu Man Zhang Ting Yao Tao Mei
Point cloud analysis faces computational system overhead, limiting its
application on mobile or edge devices. Directly employing small models may
result in a significant drop in performance since it is difficult for a small
model to adequately capture local structure and global shape information
simultaneously, which are essential clues for point cloud analysis. This paper
explores feature distillation for lightweight point cloud models. To mitigate
the semantic gap between the lightweight student and the cumbersome teacher, we
propose bidirectional knowledge reconfiguration (BKR) to distill informative
contextual knowledge from the teacher to the student. Specifically, a top-down
knowledge reconfiguration and a bottom-up knowledge reconfiguration are
developed to inherit diverse local structure information and consistent global
shape knowledge from the teacher, respectively. However, due to the farthest
point sampling in most point cloud models, the intermediate features between
teacher and student are misaligned, deteriorating the feature distillation
performance. To eliminate it, we propose a feature mover's distance (FMD) loss
based on optimal transportation, which can measure the distance between
unordered point cloud features effectively. Extensive experiments conducted on
shape classification, part segmentation, and semantic segmentation benchmarks
demonstrate the universality and superiority of our method.