Improving Real-Time Omnidirectional 3D Multi-Person Human Pose
  Estimation with People Matching and Unsupervised 2D-3D Lifting
Pawel Knap Peter Hardy Alberto Tamajo Hwasup Lim Hansung Kim
Current human pose estimation systems focus on retrieving an accurate 3D
global estimate of a single person. Therefore, this paper presents one of the
first 3D multi-person human pose estimation systems that is able to work in
real-time and is also able to handle basic forms of occlusion. First, we adjust
an off-the-shelf 2D detector and an unsupervised 2D-3D lifting model for use
with a 360$^\circ$ panoramic camera and mmWave radar sensors. We then introduce
several contributions, including camera and radar calibrations, and the
improved matching of people within the image and radar space. The system
addresses both the depth and scale ambiguity problems by employing a
lightweight 2D-3D pose lifting algorithm that is able to work in real-time
while exhibiting accurate performance in both indoor and outdoor environments
which offers both an affordable and scalable solution. Notably, our system's
time complexity remains nearly constant irrespective of the number of detected
individuals, achieving a frame rate of approximately 7-8 fps on a laptop with a
commercial-grade GPU.