Invariant Training 2D-3D Joint Hard Samples for Few-Shot Point Cloud
  Recognition
Xuanyu Yi Jiajun Deng Qianru Sun Xian-Sheng Hua Joo-Hwee Lim Hanwang Zhang
We tackle the data scarcity challenge in few-shot point cloud recognition of
3D objects by using a joint prediction from a conventional 3D model and a
well-trained 2D model. Surprisingly, such an ensemble, though seems trivial,
has hardly been shown effective in recent 2D-3D models. We find out the crux is
the less effective training for the ''joint hard samples'', which have high
confidence prediction on different wrong labels, implying that the 2D and 3D
models do not collaborate well. To this end, our proposed invariant training
strategy, called InvJoint, does not only emphasize the training more on the
hard samples, but also seeks the invariance between the conflicting 2D and 3D
ambiguous predictions. InvJoint can learn more collaborative 2D and 3D
representations for better ensemble. Extensive experiments on 3D shape
classification with widely adopted ModelNet10/40, ScanObjectNN and Toys4K, and
shape retrieval with ShapeNet-Core validate the superiority of our InvJoint.