Equivariant vs. Invariant Layers: A Comparison of Backbone and Pooling
  for Point Cloud Classification
Ashkan Shahbazi Abihith Kothapalli Xinran Liu Robert Sheng Soheil Kolouri
Learning from set-structured data, such as point clouds, has gained
significant attention from the community. Geometric deep learning provides a
blueprint for designing effective set neural networks by incorporating
permutation symmetry. Of our interest are permutation invariant networks, which
are composed of a permutation equivariant backbone, permutation invariant
global pooling, and regression/classification head. While existing literature
has focused on improving permutation equivariant backbones, the impact of
global pooling is often overlooked. In this paper, we examine the interplay
between permutation equivariant backbones and permutation invariant global
pooling on three benchmark point cloud classification datasets. Our findings
reveal that: 1) complex pooling methods, such as transport-based or
attention-based poolings, can significantly boost the performance of simple
backbones, but the benefits diminish for more complex backbones, 2) even
complex backbones can benefit from pooling layers in low data scenarios, 3)
surprisingly, the choice of pooling layers can have a more significant impact
on the model's performance than adjusting the width and depth of the backbone,
and 4) pairwise combination of pooling layers can significantly improve the
performance of a fixed backbone. Our comprehensive study provides insights for
practitioners to design better permutation invariant set neural networks.