Bidirectional Trained Tree-Structured Decoder for Handwritten
  Mathematical Expression Recognition
Hanbo Cheng Chenyu Liu Pengfei Hu Zhenrong Zhang Jiefeng Ma Jun Du
The Handwritten Mathematical Expression Recognition (HMER) task is a critical
branch in the field of OCR. Recent studies have demonstrated that incorporating
bidirectional context information significantly improves the performance of
HMER models. However, existing methods fail to effectively utilize
bidirectional context information during the inference stage. Furthermore,
current bidirectional training methods are primarily designed for string
decoders and cannot adequately generalize to tree decoders, which offer
superior generalization capabilities and structural analysis capacity. In order
to overcome these limitations, we propose the Mirror-Flipped Symbol Layout Tree
(MF-SLT) and Bidirectional Asynchronous Training (BAT) structure. Our method
extends the bidirectional training strategy to the tree decoder, allowing for
more effective training by leveraging bidirectional information. Additionally,
we analyze the impact of the visual and linguistic perception of the HMER model
separately and introduce the Shared Language Modeling (SLM) mechanism. Through
the SLM, we enhance the model's robustness and generalization when dealing with
visual ambiguity, particularly in scenarios with abundant training data. Our
approach has been validated through extensive experiments, demonstrating its
ability to achieve new state-of-the-art results on the CROHME 2014, 2016, and
2019 datasets, as well as the HME100K dataset. The code used in our experiments
will be publicly available.