ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative
  Modeling of Human-Object Interactions
Jeonghwan Kim Jisoo Kim Jeonghyeon Na Hanbyul Joo
To enable machines to learn how humans interact with the physical world in
our daily activities, it is crucial to provide rich data that encompasses the
3D motion of humans as well as the motion of objects in a learnable 3D
representation. Ideally, this data should be collected in a natural setup,
capturing the authentic dynamic 3D signals during human-object interactions. To
address this challenge, we introduce the ParaHome system, designed to capture
and parameterize dynamic 3D movements of humans and objects within a common
home environment. Our system consists of a multi-view setup with 70
synchronized RGB cameras, as well as wearable motion capture devices equipped
with an IMU-based body suit and hand motion capture gloves. By leveraging the
ParaHome system, we collect a novel large-scale dataset of human-object
interaction. Notably, our dataset offers key advancement over existing datasets
in three main aspects: (1) capturing 3D body and dexterous hand manipulation
motion alongside 3D object movement within a contextual home environment during
natural activities; (2) encompassing human interaction with multiple objects in
various episodic scenarios with corresponding descriptions in texts; (3)
including articulated objects with multiple parts expressed with parameterized
articulations. Building upon our dataset, we introduce new research tasks aimed
at building a generative model for learning and synthesizing human-object
interactions in a real-world room setting.