MoCoSA: Momentum Contrast for Knowledge Graph Completion with
  Structure-Augmented Pre-trained Language Models
Jiabang He Liu Jia Lei Wang Xiyao Li Xing Xu
Knowledge Graph Completion (KGC) aims to conduct reasoning on the facts
within knowledge graphs and automatically infer missing links. Existing methods
can mainly be categorized into structure-based or description-based. On the one
hand, structure-based methods effectively represent relational facts in
knowledge graphs using entity embeddings. However, they struggle with
semantically rich real-world entities due to limited structural information and
fail to generalize to unseen entities. On the other hand, description-based
methods leverage pre-trained language models (PLMs) to understand textual
information. They exhibit strong robustness towards unseen entities. However,
they have difficulty with larger negative sampling and often lag behind
structure-based methods. To address these issues, in this paper, we propose
Momentum Contrast for knowledge graph completion with Structure-Augmented
pre-trained language models (MoCoSA), which allows the PLM to perceive the
structural information by the adaptable structure encoder. To improve learning
efficiency, we proposed momentum hard negative and intra-relation negative
sampling. Experimental results demonstrate that our approach achieves
state-of-the-art performance in terms of mean reciprocal rank (MRR), with
improvements of 2.5% on WN18RR and 21% on OpenBG500.