UAlberta at SemEval-2023 Task 1: Context Augmentation and Translation
  for Multilingual Visual Word Sense Disambiguation
Michael Ogezi Bradley Hauer Talgat Omarov Ning Shi Grzegorz Kondrak
We describe the systems of the University of Alberta team for the
SemEval-2023 Visual Word Sense Disambiguation (V-WSD) Task. We present a novel
algorithm that leverages glosses retrieved from BabelNet, in combination with
text and image encoders. Furthermore, we compare language-specific encoders
against the application of English encoders to translated texts. As the
contexts given in the task datasets are extremely short, we also experiment
with augmenting these contexts with descriptions generated by a language model.
This yields substantial improvements in accuracy. We describe and evaluate
additional V-WSD methods which use image generation and text-conditioned image
segmentation. Overall, the results of our official submission rank us 18 out of
56 teams. Some of our unofficial results are even better than the official
ones. Our code is publicly available at https://github.com/UAlberta-NLP/v-wsd.