A Transformer-based Approach for Arabic Offline Handwritten Text
  Recognition
Saleh Momeni Bagher BabaAli
Handwriting recognition is a challenging and critical problem in the fields
of pattern recognition and machine learning, with applications spanning a wide
range of domains. In this paper, we focus on the specific issue of recognizing
offline Arabic handwritten text. Existing approaches typically utilize a
combination of convolutional neural networks for image feature extraction and
recurrent neural networks for temporal modeling, with connectionist temporal
classification used for text generation. However, these methods suffer from a
lack of parallelization due to the sequential nature of recurrent neural
networks. Furthermore, these models cannot account for linguistic rules,
necessitating the use of an external language model in the post-processing
stage to boost accuracy. To overcome these issues, we introduce two alternative
architectures, namely the Transformer Transducer and the standard
sequence-to-sequence Transformer, and compare their performance in terms of
accuracy and speed. Our approach can model language dependencies and relies
only on the attention mechanism, thereby making it more parallelizable and less
complex. We employ pre-trained Transformers for both image understanding and
language modeling. Our evaluation on the Arabic KHATT dataset demonstrates that
our proposed method outperforms the current state-of-the-art approaches for
recognizing offline Arabic handwritten text.