EAML: Ensemble Self-Attention-based Mutual Learning Network for Document
  Image Classification
Souhail Bakkali Ziheng Ming Mickael Coustaty Marçal Rusiñol
In the recent past, complex deep neural networks have received huge interest
in various document understanding tasks such as document image classification
and document retrieval. As many document types have a distinct visual style,
learning only visual features with deep CNNs to classify document images have
encountered the problem of low inter-class discrimination, and high intra-class
structural variations between its categories. In parallel, text-level
understanding jointly learned with the corresponding visual properties within a
given document image has considerably improved the classification performance
in terms of accuracy. In this paper, we design a self-attention-based fusion
module that serves as a block in our ensemble trainable network. It allows to
simultaneously learn the discriminant features of image and text modalities
throughout the training stage. Besides, we encourage mutual learning by
transferring the positive knowledge between image and text modalities during
the training stage. This constraint is realized by adding a
truncated-Kullback-Leibler divergence loss Tr-KLD-Reg as a new regularization
term, to the conventional supervised setting. To the best of our knowledge,
this is the first time to leverage a mutual learning approach along with a
self-attention-based fusion module to perform document image classification.
The experimental results illustrate the effectiveness of our approach in terms
of accuracy for the single-modal and multi-modal modalities. Thus, the proposed
ensemble self-attention-based mutual learning model outperforms the
state-of-the-art classification results based on the benchmark RVL-CDIP and
Tobacco-3482 datasets.