Word Sense Disambiguation as a Game of Neurosymbolic Darts
Tiansi Dong Rafet Sifa
Word Sense Disambiguation (WSD) is one of the hardest tasks in natural
language understanding and knowledge engineering. The glass ceiling of 80% F1
score is recently achieved through supervised deep-learning, enriched by a
variety of knowledge graphs. Here, we propose a novel neurosymbolic methodology
that is able to push the F1 score above 90%. The core of our methodology is a
neurosymbolic sense embedding, in terms of a configuration of nested balls in
n-dimensional space. The centre point of a ball well-preserves word embedding,
which partially fix the locations of balls. Inclusion relations among balls
precisely encode symbolic hypernym relations among senses, and enable simple
logic deduction among sense embeddings, which cannot be realised before. We
trained a Transformer to learn the mapping from a contextualized word embedding
to its sense ball embedding, just like playing the game of darts (a game of
shooting darts into a dartboard). A series of experiments are conducted by
utilizing pre-training n-ball embeddings, which have the coverage of around 70%
training data and 75% testing data in the benchmark WSD corpus. The F1 scores
in experiments range from 90.1% to 100.0% in all six groups of test data-sets
(each group has 4 testing data with different sizes of n-ball embeddings). Our
novel neurosymbolic methodology has the potential to break the ceiling of
deep-learning approaches for WSD. Limitations and extensions of our current
works are listed.