Transformer-Based UNet with Multi-Headed Cross-Attention Skip
  Connections to Eliminate Artifacts in Scanned Documents
David Kreuzer Michael Munz
The extraction of text in high quality is essential for text-based document
analysis tasks like Document Classification or Named Entity Recognition.
Unfortunately, this is not always ensured, as poor scan quality and the
resulting artifacts lead to errors in the Optical Character Recognition (OCR)
process. Current approaches using Convolutional Neural Networks show promising
results for background removal tasks but fail correcting artifacts like
pixelation or compression errors. For general images, Transformer backbones are
getting integrated more frequently in well-known neural network structures for
denoising tasks. In this work, a modified UNet structure using a Swin
Transformer backbone is presented to remove typical artifacts in scanned
documents. Multi-headed cross-attention skip connections are used to more
selectively learn features in respective levels of abstraction. The performance
of this approach is examined regarding compression errors, pixelation and
random noise. An improvement in text extraction quality with a reduced error
rate of up to 53.9% on the synthetic data is archived. The pretrained
base-model can be easily adapted to new artifacts. The cross-attention skip
connections allow to integrate textual information extracted from the encoder
or in form of commands to more selectively control the models outcome. The
latter is shown by means of an example application.