Unveiling the Power of Self-supervision for Multi-view Multi-human
  Association and Tracking
Wei Feng Feifan Wang Ruize Han Zekun Qian Song Wang
Multi-view multi-human association and tracking (MvMHAT), is a new but
important problem for multi-person scene video surveillance, aiming to track a
group of people over time in each view, as well as to identify the same person
across different views at the same time, which is different from previous MOT
and multi-camera MOT tasks only considering the over-time human tracking. This
way, the videos for MvMHAT require more complex annotations while containing
more information for self learning. In this work, we tackle this problem with a
self-supervised learning aware end-to-end network. Specifically, we propose to
take advantage of the spatial-temporal self-consistency rationale by
considering three properties of reflexivity, symmetry and transitivity. Besides
the reflexivity property that naturally holds, we design the self-supervised
learning losses based on the properties of symmetry and transitivity, for both
appearance feature learning and assignment matrix optimization, to associate
the multiple humans over time and across views. Furthermore, to promote the
research on MvMHAT, we build two new large-scale benchmarks for the network
training and testing of different algorithms. Extensive experiments on the
proposed benchmarks verify the effectiveness of our method. We have released
the benchmark and code to the public.