Utilizing Task-Generic Motion Prior to Recover Full-Body Motion from
  Very Sparse Signals
Myungjin Shin Dohae Lee In-Kwon Lee
The most popular type of devices used to track a user's posture in a virtual
reality experience consists of a head-mounted display and two controllers held
in both hands. However, due to the limited number of tracking sensors (three in
total), faithfully recovering the user in full-body is challenging, limiting
the potential for interactions among simulated user avatars within the virtual
world. Therefore, recent studies have attempted to reconstruct full-body poses
using neural networks that utilize previously learned human poses or accept a
series of past poses over a short period. In this paper, we propose a method
that utilizes information from a neural motion prior to improve the accuracy of
reconstructed user's motions. Our approach aims to reconstruct user's full-body
poses by predicting the latent representation of the user's overall motion from
limited input signals and integrating this information with tracking sensor
inputs. This is based on the premise that the ultimate goal of pose
reconstruction is to reconstruct the motion, which is a series of poses. Our
results show that this integration enables more accurate reconstruction of the
user's full-body motion, particularly enhancing the robustness of lower body
motion reconstruction from impoverished signals. Web:
https://https://mjsh34.github.io/mp-sspe/