Attention Where It Matters: Rethinking Visual Document Understanding
  with Selective Region Concentration
Haoyu Cao Changcun Bao Chaohu Liu Huang Chen Kun Yin Hao Liu Yinsong Liu Deqiang Jiang Xing Sun
We propose a novel end-to-end document understanding model called SeRum
(SElective Region Understanding Model) for extracting meaningful information
from document images, including document analysis, retrieval, and office
automation.
  Unlike state-of-the-art approaches that rely on multi-stage technical schemes
and are computationally expensive,
  SeRum converts document image understanding and recognition tasks into a
local decoding process of the visual tokens of interest, using a content-aware
token merge module.
  This mechanism enables the model to pay more attention to regions of interest
generated by the query decoder, improving the model's effectiveness and
speeding up the decoding speed of the generative scheme.
  We also designed several pre-training tasks to enhance the understanding and
local awareness of the model.
  Experimental results demonstrate that SeRum achieves state-of-the-art
performance on document understanding tasks and competitive results on text
spotting tasks.
  SeRum represents a substantial advancement towards enabling efficient and
effective end-to-end document understanding.