Multi-task 3D building understanding with multi-modal pretraining
Shicheng Xu
This paper explores various learning strategies for 3D building type
classification and part segmentation on the BuildingNet dataset. ULIP with
PointNeXt and PointNeXt segmentation are extended for the classification and
segmentation task on BuildingNet dataset. The best multi-task PointNeXt-s model
with multi-modal pretraining achieves 59.36 overall accuracy for 3D building
type classification, and 31.68 PartIoU for 3D building part segmentation on
validation split. The final PointNeXt XL model achieves 31.33 PartIoU and 22.78
ShapeIoU on test split for BuildingNet-Points segmentation, which significantly
improved over PointNet++ model reported from BuildingNet paper, and it won the
1st place in the BuildingNet challenge at CVPR23 StruCo3D workshop.