SM$^3$: Self-Supervised Multi-task Modeling with Multi-view 2D Images
  for Articulated Objects
Haowen Wang Zhen Zhao Zhao Jin Zhengping Che Liang Qiao Yakun Huang Zhipeng Fan Xiuquan Qiao Jian Tang
Reconstructing real-world objects and estimating their movable joint
structures are pivotal technologies within the field of robotics. Previous
research has predominantly focused on supervised approaches, relying on
extensively annotated datasets to model articulated objects within limited
categories. However, this approach falls short of effectively addressing the
diversity present in the real world. To tackle this issue, we propose a
self-supervised interaction perception method, referred to as SM$^3$, which
leverages multi-view RGB images captured before and after interaction to model
articulated objects, identify the movable parts, and infer the parameters of
their rotating joints. By constructing 3D geometries and textures from the
captured 2D images, SM$^3$ achieves integrated optimization of movable part and
joint parameters during the reconstruction process, obviating the need for
annotations. Furthermore, we introduce the MMArt dataset, an extension of
PartNet-Mobility, encompassing multi-view and multi-modal data of articulated
objects spanning diverse categories. Evaluations demonstrate that SM$^3$
surpasses existing benchmarks across various categories and objects, while its
adaptability in real-world scenarios has been thoroughly validated.