Multi-View Person Matching and 3D Pose Estimation with Arbitrary
  Uncalibrated Camera Networks
Yan Xu Kris Kitani
Cross-view person matching and 3D human pose estimation in multi-camera
networks are particularly difficult when the cameras are extrinsically
uncalibrated. Existing efforts generally require large amounts of 3D data for
training neural networks or known camera poses for geometric constraints to
solve the problem. However, camera poses and 3D data annotation are usually
expensive and not always available. We present a method, PME, that solves the
two tasks without requiring either information. Our idea is to address
cross-view person matching as a clustering problem using each person as a
cluster center, then obtain correspondences from person matches, and estimate
3D human poses through multi-view triangulation and bundle adjustment. We solve
the clustering problem by introducing a "size constraint" using the number of
cameras and a "source constraint" using the fact that two people from the same
camera view should not match, to narrow the solution space to a small feasible
region. The 2D human poses used in clustering are obtained through a
pre-trained 2D pose detector, so our method does not require expensive 3D
training data for each new scene. We extensively evaluate our method on three
open datasets and two indoor and outdoor datasets collected using arbitrarily
set cameras. Our method outperforms other methods by a large margin on
cross-view person matching, reaches SOTA performance on 3D human pose
estimation without using either camera poses or 3D training data, and shows
good generalization ability across five datasets of various environment
settings.