TWIG: Towards pre-hoc Hyperparameter Optimisation and Cross-Graph
  Generalisation via Simulated KGE Models
Jeffrey Sardina John D. Kelleher Declan O'Sullivan
In this paper we introduce TWIG (Topologically-Weighted Intelligence
Generation), a novel, embedding-free paradigm for simulating the output of KGEs
that uses a tiny fraction of the parameters. TWIG learns weights from inputs
that consist of topological features of the graph data, with no coding for
latent representations of entities or edges. Our experiments on the UMLS
dataset show that a single TWIG neural network can predict the results of
state-of-the-art ComplEx-N3 KGE model nearly exactly on across all
hyperparameter configurations. To do this it uses a total of 2590 learnable
parameters, but accurately predicts the results of 1215 different
hyperparameter combinations with a combined cost of 29,322,000 parameters.
Based on these results, we make two claims: 1) that KGEs do not learn latent
semantics, but only latent representations of structural patterns; 2) that
hyperparameter choice in KGEs is a deterministic function of the KGE model and
graph structure. We further hypothesise that, as TWIG can simulate KGEs without
embeddings, that node and edge embeddings are not needed to learn to accurately
predict new facts in KGs. Finally, we formulate all of our findings under the
umbrella of the ``Structural Generalisation Hypothesis", which suggests that
``twiggy" embedding-free / data-structure-based learning methods can allow a
single neural network to simulate KGE performance, and perhaps solve the Link
Prediction task, across many KGs from diverse domains and with different
semantics.