MSFormer: A Skeleton-multiview Fusion Method For Tooth Instance
  Segmentation
Yuan Li Huan Liu Yubo Tao Xiangyang He Haifeng Li Xiaohu Guo Hai Lin
Recently, deep learning-based tooth segmentation methods have been limited by
the expensive and time-consuming processes of data collection and labeling.
Achieving high-precision segmentation with limited datasets is critical. A
viable solution to this entails fine-tuning pre-trained multiview-based models,
thereby enhancing performance with limited data. However, relying solely on
two-dimensional (2D) images for three-dimensional (3D) tooth segmentation can
produce suboptimal outcomes because of occlusion and deformation, i.e.,
incomplete and distorted shape perception. To improve this fine-tuning-based
solution, this paper advocates 2D-3D joint perception. The fundamental
challenge in employing 2D-3D joint perception with limited data is that the
3D-related inputs and modules must follow a lightweight policy instead of using
huge 3D data and parameter-rich modules that require extensive training data.
Following this lightweight policy, this paper selects skeletons as the 3D
inputs and introduces MSFormer, a novel method for tooth segmentation. MSFormer
incorporates two lightweight modules into existing multiview-based models: a
3D-skeleton perception module to extract 3D perception from skeletons and a
skeleton-image contrastive learning module to obtain the 2D-3D joint perception
by fusing both multiview and skeleton perceptions. The experimental results
reveal that MSFormer paired with large pre-trained multiview models achieves
state-of-the-art performance, requiring only 100 training meshes. Furthermore,
the segmentation accuracy is improved by 2.4%-5.5% with the increasing volume
of training data.