Primitive-based 3D Human-Object Interaction Modelling and Programming
Siqi Liu Yong-Lu Li Zhou Fang Xinpeng Liu Yang You Cewu Lu
Embedding Human and Articulated Object Interaction (HAOI) in 3D is an
important direction for a deeper human activity understanding. Different from
previous works that use parametric and CAD models to represent humans and
objects, in this work, we propose a novel 3D geometric primitive-based language
to encode both humans and objects. Given our new paradigm, humans and objects
are all compositions of primitives instead of heterogeneous entities. Thus,
mutual information learning may be achieved between the limited 3D data of
humans and different object categories. Moreover, considering the simplicity of
the expression and the richness of the information it contains, we choose the
superquadric as the primitive representation. To explore an effective embedding
of HAOI for the machine, we build a new benchmark on 3D HAOI consisting of
primitives together with their images and propose a task requiring machines to
recover 3D HAOI using primitives from images. Moreover, we propose a baseline
of single-view 3D reconstruction on HAOI. We believe this primitive-based 3D
HAOI representation would pave the way for 3D HAOI studies. Our code and data
are available at https://mvig-rhos.com/p3haoi.