Self-supervised Learning of Rotation-invariant 3D Point Set Features
  using Transformer and its Self-distillation
Takahiko Furuya Zhoujie Chen Ryutarou Ohbuchi Zhenzhong Kuang
Invariance against rotations of 3D objects is an important property in
analyzing 3D point set data. Conventional 3D point set DNNs having rotation
invariance typically obtain accurate 3D shape features via supervised learning
by using labeled 3D point sets as training samples. However, due to the rapid
increase in 3D point set data and the high cost of labeling, a framework to
learn rotation-invariant 3D shape features from numerous unlabeled 3D point
sets is required. This paper proposes a novel self-supervised learning
framework for acquiring accurate and rotation-invariant 3D point set features
at object-level. Our proposed lightweight DNN architecture decomposes an input
3D point set into multiple global-scale regions, called tokens, that preserve
the spatial layout of partial shapes composing the 3D object. We employ a
self-attention mechanism to refine the tokens and aggregate them into an
expressive rotation-invariant feature per 3D point set. Our DNN is effectively
trained by using pseudo-labels generated by a self-distillation framework. To
facilitate the learning of accurate features, we propose to combine multi-crop
and cut-mix data augmentation techniques to diversify 3D point sets for
training. Through a comprehensive evaluation, we empirically demonstrate that,
(1) existing rotation-invariant DNN architectures designed for supervised
learning do not necessarily learn accurate 3D shape features under a
self-supervised learning scenario, and (2) our proposed algorithm learns
rotation-invariant 3D point set features that are more accurate than those
learned by existing algorithms. Code will be available at
https://github.com/takahikof/RIPT_SDMM