Full-Body Motion Reconstruction with Sparse Sensing from Graph
  Perspective
Feiyu Yao Zongkai Wu Li Yi
Estimating 3D full-body pose from sparse sensor data is a pivotal technique
employed for the reconstruction of realistic human motions in Augmented Reality
and Virtual Reality. However, translating sparse sensor signals into
comprehensive human motion remains a challenge since the sparsely distributed
sensors in common VR systems fail to capture the motion of full human body. In
this paper, we use well-designed Body Pose Graph (BPG) to represent the human
body and translate the challenge into a prediction problem of graph missing
nodes. Then, we propose a novel full-body motion reconstruction framework based
on BPG. To establish BPG, nodes are initially endowed with features extracted
from sparse sensor signals. Features from identifiable joint nodes across
diverse sensors are amalgamated and processed from both temporal and spatial
perspectives. Temporal dynamics are captured using the Temporal Pyramid
Structure, while spatial relations in joint movements inform the spatial
attributes. The resultant features serve as the foundational elements of the
BPG nodes. To further refine the BPG, node features are updated through a graph
neural network that incorporates edge reflecting varying joint relations. Our
method's effectiveness is evidenced by the attained state-of-the-art
performance, particularly in lower body motion, outperforming other baseline
methods. Additionally, an ablation study validates the efficacy of each module
in our proposed framework.