TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition
Tianlun Zheng Zhineng Chen Jinfeng Bai Hongtao Xie Yu-Gang Jiang
Text irregularities pose significant challenges to scene text recognizers.
Thin-Plate Spline (TPS)-based rectification is widely regarded as an effective
means to deal with them. Currently, the calculation of TPS transformation
parameters purely depends on the quality of regressed text borders. It ignores
the text content and often leads to unsatisfactory rectified results for
severely distorted text. In this work, we introduce TPS++, an
attention-enhanced TPS transformation that incorporates the attention mechanism
to text rectification for the first time. TPS++ formulates the parameter
calculation as a joint process of foreground control point regression and
content-based attention score estimation, which is computed by a dedicated
designed gated-attention block. TPS++ builds a more flexible content-aware
rectifier, generating a natural text correction that is easier to read by the
subsequent recognizer. Moreover, TPS++ shares the feature backbone with the
recognizer in part and implements the rectification at feature-level rather
than image-level, incurring only a small overhead in terms of parameters and
inference time. Experiments on public benchmarks show that TPS++ consistently
improves the recognition and achieves state-of-the-art accuracy. Meanwhile, it
generalizes well on different backbones and recognizers. Code is at
https://github.com/simplify23/TPS_PP.