MeT: A Graph Transformer for Semantic Segmentation of 3D Meshes
Giuseppe Vecchio Luca Prezzavento Carmelo Pino Francesco Rundo Simone Palazzo Concetto Spampinato
Polygonal meshes have become the standard for discretely approximating 3D
shapes, thanks to their efficiency and high flexibility in capturing
non-uniform shapes. This non-uniformity, however, leads to irregularity in the
mesh structure, making tasks like segmentation of 3D meshes particularly
challenging. Semantic segmentation of 3D mesh has been typically addressed
through CNN-based approaches, leading to good accuracy. Recently, transformers
have gained enough momentum both in NLP and computer vision fields, achieving
performance at least on par with CNN models, supporting the long-sought
architecture universalism. Following this trend, we propose a transformer-based
method for semantic segmentation of 3D mesh motivated by a better modeling of
the graph structure of meshes, by means of global attention mechanisms. In
order to address the limitations of standard transformer architectures in
modeling relative positions of non-sequential data, as in the case of 3D
meshes, as well as in capturing the local context, we perform positional
encoding by means the Laplacian eigenvectors of the adjacency matrix, replacing
the traditional sinusoidal positional encodings, and by introducing
clustering-based features into the self-attention and cross-attention
operators. Experimental results, carried out on three sets of the Shape COSEG
Dataset, on the human segmentation dataset proposed in Maron et al., 2017 and
on the ShapeNet benchmark, show how the proposed approach yields
state-of-the-art performance on semantic segmentation of 3D meshes.