Spatio-temporal MLP-graph network for 3D human pose estimation
Tanvir Hassan A. Ben Hamza
Graph convolutional networks and their variants have shown significant
promise in 3D human pose estimation. Despite their success, most of these
methods only consider spatial correlations between body joints and do not take
into account temporal correlations, thereby limiting their ability to capture
relationships in the presence of occlusions and inherent ambiguity. To address
this potential weakness, we propose a spatio-temporal network architecture
composed of a joint-mixing multi-layer perceptron block that facilitates
communication among different joints and a graph weighted Jacobi network block
that enables communication among various feature channels. The major novelty of
our approach lies in a new weighted Jacobi feature propagation rule obtained
through graph filtering with implicit fairing. We leverage temporal information
from the 2D pose sequences, and integrate weight modulation into the model to
enable untangling of the feature transformations of distinct nodes. We also
employ adjacency modulation with the aim of learning meaningful correlations
beyond defined linkages between body joints by altering the graph topology
through a learnable modulation matrix. Extensive experiments on two benchmark
datasets demonstrate the effectiveness of our model, outperforming recent
state-of-the-art methods for 3D human pose estimation.