MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body
  Reconstruction
Yitao Zhu Sheng Wang Mengjie Xu Zixu Zhuang Zhixin Wang Kaidong Wang Han Zhang Qian Wang
Multiple cameras can provide multi-view video coverage of a person. It is
necessary to fuse multi-view data, e.g., for subsequent behavioral analysis,
while such fusion often relies on calibration of cameras in traditional
solutions. However, it is non-trivial to calibrate multiple cameras. In this
work, we propose a method to reconstruct 3D human body from multiple
uncalibrated camera views. First, we adopt a pre-trained human body encoder to
process each individual camera view, such that human body models and parameters
can be reconstructed for each view. Next, instead of simply averaging models
across views, we train a network to determine the weights of individual views
for their fusion, based on the parameters estimated for joints and hands of
human body as well as camera positions. Further, we turn to the mesh surface of
human body for dynamic fusion, such that facial expression can be seamlessly
integrated into the model of human body. Our method has demonstrated superior
performance in reconstructing human body upon two public datasets. More
importantly, our method can flexibly support ad-hoc deployment of an arbitrary
number of cameras, which has significant potential in related applications. We
will release source code upon acceptance of the paper.