Select and Reorder: A Novel Approach for Neural Sign Language Production
Harry Walsh Ben Saunders Richard Bowden
Sign languages, often categorised as low-resource languages, face significant
challenges in achieving accurate translation due to the scarcity of parallel
annotated datasets. This paper introduces Select and Reorder (S&R), a novel
approach that addresses data scarcity by breaking down the translation process
into two distinct steps: Gloss Selection (GS) and Gloss Reordering (GR). Our
method leverages large spoken language models and the substantial lexical
overlap between source spoken languages and target sign languages to establish
an initial alignment. Both steps make use of Non-AutoRegressive (NAR) decoding
for reduced computation and faster inference speeds. Through this
disentanglement of tasks, we achieve state-of-the-art BLEU and Rouge scores on
the Meine DGS Annotated (mDGS) dataset, demonstrating a substantial BLUE-1
improvement of 37.88% in Text to Gloss (T2G) Translation. This innovative
approach paves the way for more effective translation models for sign
languages, even in resource-constrained settings.