3DHR-Co: A Collaborative Test-time Refinement Framework for In-the-Wild
  3D Human-Body Reconstruction Task
Jonathan Samuel Lumentut Kyoung Mu Lee
The field of 3D human-body reconstruction (abbreviated as 3DHR) that utilizes
parametric pose and shape representations has witnessed significant
advancements in recent years. However, the application of 3DHR techniques to
handle real-world, diverse scenes, known as in-the-wild data, still faces
limitations. The primary challenge arises as curating accurate 3D human pose
ground truth (GT) for in-the-wild scenes is still difficult to obtain due to
various factors. Recent test-time refinement approaches on 3DHR leverage
initial 2D off-the-shelf human keypoints information to support the lack of 3D
supervision on in-the-wild data. However, we observed that additional 2D
supervision alone could cause the overfitting issue on common 3DHR backbones,
making the 3DHR test-time refinement task seem intractable. We answer this
challenge by proposing a strategy that complements 3DHR test-time refinement
work under a collaborative approach. Specifically, we initially apply a
pre-adaptation approach that works by collaborating various 3DHR models in a
single framework to directly improve their initial outputs. This approach is
then further combined with the test-time adaptation work under specific
settings that minimize the overfitting issue to further boost the 3DHR
performance. The whole framework is termed as 3DHR-Co, and on the experiment
sides, we showed that the proposed work can significantly enhance the scores of
common classic 3DHR backbones up to -34 mm pose error suppression, putting them
among the top list on the in-the-wild benchmark data. Such achievement shows
that our approach helps unveil the true potential of the common classic 3DHR
backbones. Based on these findings, we further investigate various settings on
the proposed framework to better elaborate the capability of our collaborative
approach in the 3DHR task.