Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation
Ti Wang Mengyuan Liu Hong Liu Bin Ren Yingxuan You Wenhao Li Nicu Sebe Xia Li
Although data-driven methods have achieved success in 3D human pose
estimation, they often suffer from domain gaps and exhibit limited
generalization. In contrast, optimization-based methods excel in fine-tuning
for specific cases but are generally inferior to data-driven methods in overall
performance. We observe that previous optimization-based methods commonly rely
on projection constraint, which only ensures alignment in 2D space, potentially
leading to the overfitting problem. To address this, we propose an
Uncertainty-Aware testing-time Optimization (UAO) framework, which keeps the
prior information of pre-trained model and alleviates the overfitting problem
using the uncertainty of joints. Specifically, during the training phase, we
design an effective 2D-to-3D network for estimating the corresponding 3D pose
while quantifying the uncertainty of each 3D joint. For optimization during
testing, the proposed optimization framework freezes the pre-trained model and
optimizes only a latent state. Projection loss is then employed to ensure the
generated poses are well aligned in 2D space for high-quality optimization.
Furthermore, we utilize the uncertainty of each joint to determine how much
each joint is allowed for optimization. The effectiveness and superiority of
the proposed framework are validated through extensive experiments on two
challenging datasets: Human3.6M and MPI-INF-3DHP. Notably, our approach
outperforms the previous best result by a large margin of 4.5% on Human3.6M.
Our source code will be open-sourced.