Hierarchical Mutual Information Analysis: Towards Multi-view Clustering
  in The Wild
Jiatai Wang Zhiwei Xu Xuewen Yang Xin Wang
Multi-view clustering (MVC) can explore common semantics from unsupervised
views generated by different sources, and thus has been extensively used in
applications of practical computer vision. Due to the spatio-temporal
asynchronism, multi-view data often suffer from view missing and are unaligned
in real-world applications, which makes it difficult to learn consistent
representations. To address the above issues, this work proposes a deep MVC
framework where data recovery and alignment are fused in a hierarchically
consistent way to maximize the mutual information among different views and
ensure the consistency of their latent spaces. More specifically, we first
leverage dual prediction to fill in missing views while achieving the
instance-level alignment, and then take the contrastive reconstruction to
achieve the class-level alignment. To the best of our knowledge, this could be
the first successful attempt to handle the missing and unaligned data problem
separately with different learning paradigms. Extensive experiments on public
datasets demonstrate that our method significantly outperforms state-of-the-art
methods on multi-view clustering even in the cases of view missing and
unalignment.