Why are hyperbolic neural networks effective? A study on hierarchical
  representation capability
Shicheng Tan Huanjing Zhao Shu Zhao Yanping Zhang
Hyperbolic Neural Networks (HNNs), operating in hyperbolic space, have been
widely applied in recent years, motivated by the existence of an optimal
embedding in hyperbolic space that can preserve data hierarchical relationships
(termed Hierarchical Representation Capability, HRC) more accurately than
Euclidean space. However, there is no evidence to suggest that HNNs can achieve
this theoretical optimal embedding, leading to much research being built on
flawed motivations. In this paper, we propose a benchmark for evaluating HRC
and conduct a comprehensive analysis of why HNNs are effective through
large-scale experiments. Inspired by the analysis results, we propose several
pre-training strategies to enhance HRC and improve the performance of
downstream tasks, further validating the reliability of the analysis.
Experiments show that HNNs cannot achieve the theoretical optimal embedding.
The HRC is significantly affected by the optimization objectives and
hierarchical structures, and enhancing HRC through pre-training strategies can
significantly improve the performance of HNNs.