Feature Learning in Image Hierarchies using Functional Maximal
  Correlation
Bo Hu Yuheng Bu José C. Príncipe
This paper proposes the Hierarchical Functional Maximal Correlation Algorithm
(HFMCA), a hierarchical methodology that characterizes dependencies across two
hierarchical levels in multiview systems. By framing view similarities as
dependencies and ensuring contrastivity by imposing orthonormality, HFMCA
achieves faster convergence and increased stability in self-supervised
learning. HFMCA defines and measures dependencies within image hierarchies,
from pixels and patches to full images. We find that the network topology for
approximating orthonormal basis functions aligns with a vanilla CNN, enabling
the decomposition of density ratios between neighboring layers of feature maps.
This approach provides powerful interpretability, revealing the resemblance
between supervision and self-supervision through the lens of internal
representations.