Advancing Text-to-GLOSS Neural Translation Using a Novel Hyper-parameter
  Optimization Technique
Younes Ouargani Noussaima El Khattabi
In this paper, we investigate the use of transformers for Neural Machine
Translation of text-to-GLOSS for Deaf and Hard-of-Hearing communication. Due to
the scarcity of available data and limited resources for text-to-GLOSS
translation, we treat the problem as a low-resource language task. We use our
novel hyper-parameter exploration technique to explore a variety of
architectural parameters and build an optimal transformer-based architecture
specifically tailored for text-to-GLOSS translation. The study aims to improve
the accuracy and fluency of Neural Machine Translation generated GLOSS. This is
achieved by examining various architectural parameters including layer count,
attention heads, embedding dimension, dropout, and label smoothing to identify
the optimal architecture for improving text-to-GLOSS translation performance.
The experiments conducted on the PHOENIX14T dataset reveal that the optimal
transformer architecture outperforms previous work on the same dataset. The
best model reaches a ROUGE (Recall-Oriented Understudy for Gisting Evaluation)
score of 55.18% and a BLEU-1 (BiLingual Evaluation Understudy 1) score of
63.6%, outperforming state-of-the-art results on the BLEU1 and ROUGE score by
8.42 and 0.63 respectively.