Entity or Relation Embeddings? An Analysis of Encoding Strategies for
  Relation Extraction
Frank Mtumbuka Steven Schockaert
Relation extraction is essentially a text classification problem, which can
be tackled by fine-tuning a pre-trained language model (LM). However, a key
challenge arises from the fact that relation extraction cannot
straightforwardly be reduced to sequence or token classification. Existing
approaches therefore solve the problem in an indirect way: they fine-tune an LM
to learn embeddings of the head and tail entities, and then predict the
relationship from these entity embeddings. Our hypothesis in this paper is that
relation extraction models can be improved by capturing relationships in a more
direct way. In particular, we experiment with appending a prompt with a [MASK]
token, whose contextualised representation is treated as a relation embedding.
While, on its own, this strategy significantly underperforms the aforementioned
approach, we find that the resulting relation embeddings are highly
complementary to what is captured by embeddings of the head and tail entity. By
jointly considering both types of representations, we end up with a simple
model that outperforms the state-of-the-art across several relation extraction
benchmarks.