Enhanced Spatio-Temporal Context for Temporally Consistent Robust 3D
  Human Motion Recovery from Monocular Videos
Sushovan Chanda Amogh Tiwari Lokender Tiwari Brojeshwar Bhowmick Avinash Sharma Hrishav Barua
Recovering temporally consistent 3D human body pose, shape and motion from a
monocular video is a challenging task due to (self-)occlusions, poor lighting
conditions, complex articulated body poses, depth ambiguity, and limited
availability of annotated data. Further, doing a simple perframe estimation is
insufficient as it leads to jittery and implausible results. In this paper, we
propose a novel method for temporally consistent motion estimation from a
monocular video. Instead of using generic ResNet-like features, our method uses
a body-aware feature representation and an independent per-frame pose and
camera initialization over a temporal window followed by a novel
spatio-temporal feature aggregation by using a combination of self-similarity
and self-attention over the body-aware features and the perframe
initialization. Together, they yield enhanced spatiotemporal context for every
frame by considering remaining past and future frames. These features are used
to predict the pose and shape parameters of the human body model, which are
further refined using an LSTM. Experimental results on the publicly available
benchmark data show that our method attains significantly lower acceleration
error and outperforms the existing state-of-the-art methods over all key
quantitative evaluation metrics, including complex scenarios like partial
occlusion, complex poses and even relatively low illumination.