Relation-Aware Distribution Representation Network for Person Clustering
  with Multiple Modalities
Kaijian Liu Shixiang Tang Ziyue Li Zhishuai Li Lei Bai Feng Zhu Rui Zhao
Person clustering with multi-modal clues, including faces, bodies, and
voices, is critical for various tasks, such as movie parsing and identity-based
movie editing. Related methods such as multi-view clustering mainly project
multi-modal features into a joint feature space. However, multi-modal clue
features are usually rather weakly correlated due to the semantic gap from the
modality-specific uniqueness. As a result, these methods are not suitable for
person clustering. In this paper, we propose a Relation-Aware Distribution
representation Network (RAD-Net) to generate a distribution representation for
multi-modal clues. The distribution representation of a clue is a vector
consisting of the relation between this clue and all other clues from all
modalities, thus being modality agnostic and good for person clustering.
Accordingly, we introduce a graph-based method to construct distribution
representation and employ a cyclic update policy to refine distribution
representation progressively. Our method achieves substantial improvements of
+6% and +8.2% in F-score on the Video Person-Clustering Dataset (VPCD) and
VoxCeleb2 multi-view clustering dataset, respectively. Codes will be released
publicly upon acceptance.