We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations. In this paper, we present a new approach for knowledge graph embedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry, inversion, and composition. Specifically, the RotatE model defines each relation as a rotation from the source entity to the target entity in the complex vector space. In addition, we propose a novel self-adversarial negative sampling technique for efficiently and effectively training the RotatE model. Experimental results on multiple benchmark knowledge graphs show that the proposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction. Knowledge graphs are collections of factual triplets, where each triplet (h, r, t) represents a relation r between a head entity h and a tail entity t. Examples of real-world knowledge graphs include Freebase BID0 , Yago (Suchanek et al., 2007) , and WordNet (Miller, 1995) . Knowledge graphs are potentially useful to a variety of applications such as question-answering BID10 , information retrieval BID30 , recommender systems BID34 , and natural language processing BID31 . Research on knowledge graphs is attracting growing interests in both academia and industry communities.Since knowledge graphs are usually incomplete, a fundamental problem for knowledge graph is predicting the missing links. Recently, extensive studies have been done on learning low-dimensional representations of entities and relations for missing link prediction (a.k.a., knowledge graph embedding) BID3 BID28 BID7 . These methods have been shown to be scalable and effective. The general intuition of these methods is to model and infer the connectivity patterns in knowledge graphs according to the observed knowledge facts. For example, some relations are symmetric (e.g., marriage) while others are antisymmetric (e.g., filiation); some relations are the inverse of other relations (e.g., hypernym and hyponym); and some relations may be composed by others (e.g., my mother's husband is my father). It is critical to find ways to model and infer these patterns, i.e., symmetry/antisymmetry, inversion, and composition, from the observed facts in order to predict missing links.Indeed, many existing approaches have been trying to either implicitly or explicitly model one or a few of the above relation patterns BID3 BID29 BID17 Table 1: The score functions f r (h, t) of several knowledge graph embedding models, where · denotes the generalized dot product, • denotes the Hadamard product, ⊗ denotes circular correlation, σ denotes activation function and * denotes 2D convolution. · denotes conjugate for complex vectors, and 2D reshaping for real vectors in ConvE model. TransX represents a wide range of TransE's variants, such as TransH BID29 , TransR BID17 , and STransE BID23 , where g r,i (·) denotes a matrix multiplication with respect to relation r. BID32 BID28 . For example, the TransE model BID2 , which represents relations as translations, aims to model the inversion and composition patterns; the DisMult model BID32 , which models the three-way interactions between head entities, relations, and tail entities, aims to model the symmetry pattern. However, none of existing models is capable of modeling and inferring all the above patterns. Therefore, we are looking for an approach that is able to model and infer all the three types of relation patterns.In this paper, we propose such an approach called RotatE for knowledge graph embedding. Our motivation is from Euler's identity e iθ = cos θ + i sin θ, which indicates that a unitary complex number can be regarded as a rotation in the complex plane. Specifically, the RotatE model maps the entities and relations to the complex vector space and defines each relation as a rotation from the source entity to the target entity. Given a triplet (h, r, t), we expect that t = h • r, where h, r, t ∈ C k are the embeddings, the modulus |r i | = 1 and • denotes the Hadamard (element-wise) product. Specifically, for each dimension in the complex space, we expect that:t i = h i r i , where h i , r i , t i ∈ C and |r i | = 1.It turns out that such a simple operation can effectively model all the three relation patterns: symmetric/antisymmetric, inversion, and composition. For example, a relation r is symmetric if and only if each element of its embedding r, i.e. r i , satisfies r i = e 0/iπ = ±1; two relations r 1 and r 2 are inverse if and only if their embeddings are conjugates: r 2 =r 1 ; a relation r 3 = e iθ3 is a combination of other two relations r 1 = e iθ1 and r 2 = e iθ2 if and only if r 3 = r 1 • r 2 (i.e. θ 3 = θ 1 + θ 2 ). Moreover, the RotatE model is scalable to large knowledge graphs as it remains linear in both time and memory.To effectively optimizing the RotatE, we further propose a novel self-adversarial negative sampling technique, which generates negative samples according to the current entity and relation embeddings. The proposed technique is very general and can be applied to many existing knowledge graph embedding models. We evaluate the RotatE on four large knowledge graph benchmark datasets including FB15k BID3 , WN18 BID3 ), FB15k-237 (Toutanova & Chen, 2015 and WN18RR BID7 . Experimental results show that the RotatE model significantly outperforms existing state-of-the-art approaches. In addition, RotatE also outperforms state-of-the-art models on Countries BID4 , a benchmark explicitly designed for composition pattern inference and modeling. To the best of our knowledge, RotatE is the first model that achieves state-of-the-art performance on all the benchmarks. 2 The p-norm of a complex vector v is defined as v p = p |vi| p . We use L1-norm for all distancebased models in this paper and drop the subscript of · 1 for brevity. We have proposed a new knowledge graph embedding method called RotatE, which represents entities as complex vectors and relations as rotations in complex vector space. In addition, we propose a novel self-adversarial negative sampling technique for efficiently and effectively training the RotatE model. Our experimental results show that the RotatE model outperforms all existing state-of-theart models on four large-scale benchmarks. Moreover, RotatE also achieves state-of-the-art results on a benchmark that is explicitly designed for composition pattern inference and modeling. A deep investigation into RotatE relation embeddings shows that the three relation patterns are implicitly represented in the relation embeddings. In the future, we plan to evaluate the RotatE model on more datasets and leverage a probabilistic framework to model the uncertainties of entities and relations. No existing models are capable of modeling all the three relation patterns. For example, TransE cannot model the symmetry pattern because it would yield r = 0 for symmetric relations; TransX can infer and model the symmetry/antisymmetry pattern when g r,1 = g r,2 , e.g. in TransH BID29 , but cannot infer inversion and composition as g r,1 and g r,2 are invertible matrix multiplications; due to its symmetric nature, DistMult is difficult to model the asymmetric and inversion pattern; ComplEx addresses the problem of DisMult and is able to infer both the symmetry and asymmetric patterns with complex embeddings. Moreover, it can infer inversion rules because the complex conjugate of the solution to arg max r Re( x, r, y ) is exactly the solution to arg max r Re( y, r, x ). However, ComplEx cannot infer composition rules, since it does not model a bijection mapping from h to t via relation r. These concerns are summarized in TAB0 .B PROOF OF LEMMA 1Proof. if r(x, y) and r(y, x) hold, we have y = r • x ∧ x = r • y ⇒ r • r = 1 Otherwise, if r(x, y) and ¬r(y, x) hold, we have DISPLAYFORM0 Proof. if r 1 (x, y) and r 2 (y, x) hold, we have DISPLAYFORM1 Proof. if r 1 (x, z) , r 2 (x, y ) and r 3 (y, z) hold, we have DISPLAYFORM2
A new state-of-the-art approach for knowledge graph embedding. Presents a neural link prediction scoring function that can infer symmetry, anti-symmetry, inversion and composition patterns of relations in a knowledge base. This paper proposes an approach to knowledge graph embedding by modeling relations as rotations in the complex vector space. Proposes a method for graph embedding to be used for link prediction