Self-Supervised 3D Scene Flow Estimation and Motion Prediction using
  Local Rigidity Prior
Ruibo Li Chi Zhang Zhe Wang Chunhua Shen Guosheng Lin
In this article, we investigate self-supervised 3D scene flow estimation and
class-agnostic motion prediction on point clouds. A realistic scene can be well
modeled as a collection of rigidly moving parts, therefore its scene flow can
be represented as a combination of the rigid motion of these individual parts.
Building upon this observation, we propose to generate pseudo scene flow labels
for self-supervised learning through piecewise rigid motion estimation, in
which the source point cloud is decomposed into local regions and each region
is treated as rigid. By rigidly aligning each region with its potential
counterpart in the target point cloud, we obtain a region-specific rigid
transformation to generate its pseudo flow labels. To mitigate the impact of
potential outliers on label generation, when solving the rigid registration for
each region, we alternately perform three steps: establishing point
correspondences, measuring the confidence for the correspondences, and updating
the rigid transformation based on the correspondences and their confidence. As
a result, confident correspondences will dominate label generation and a
validity mask will be derived for the generated pseudo labels. By using the
pseudo labels together with their validity mask for supervision, models can be
trained in a self-supervised manner. Extensive experiments on FlyingThings3D
and KITTI datasets demonstrate that our method achieves new state-of-the-art
performance in self-supervised scene flow learning, without any ground truth
scene flow for supervision, even performing better than some supervised
counterparts. Additionally, our method is further extended to class-agnostic
motion prediction and significantly outperforms previous state-of-the-art
self-supervised methods on nuScenes dataset.