Character Queries: A Transformer-based Approach to On-Line Handwritten
  Character Segmentation
Michael Jungo Beat Wolf Andrii Maksai Claudiu Musat Andreas Fischer
On-line handwritten character segmentation is often associated with
handwriting recognition and even though recognition models include mechanisms
to locate relevant positions during the recognition process, it is typically
insufficient to produce a precise segmentation. Decoupling the segmentation
from the recognition unlocks the potential to further utilize the result of the
recognition. We specifically focus on the scenario where the transcription is
known beforehand, in which case the character segmentation becomes an
assignment problem between sampling points of the stylus trajectory and
characters in the text. Inspired by the $k$-means clustering algorithm, we view
it from the perspective of cluster assignment and present a Transformer-based
architecture where each cluster is formed based on a learned character query in
the Transformer decoder block. In order to assess the quality of our approach,
we create character segmentation ground truths for two popular on-line
handwriting datasets, IAM-OnDB and HANDS-VNOnDB, and evaluate multiple methods
on them, demonstrating that our approach achieves the overall best results.