M3-VRD: Multimodal Multi-task Multi-teacher Visually-Rich Form Document
  Understanding
Yihao Ding Lorenzo Vaiani Caren Han Jean Lee Paolo Garza Josiah Poon Luca Cagliero
This paper presents a groundbreaking multimodal, multi-task, multi-teacher
joint-grained knowledge distillation model for visually-rich form document
understanding. The model is designed to leverage insights from both
fine-grained and coarse-grained levels by facilitating a nuanced correlation
between token and entity representations, addressing the complexities inherent
in form documents. Additionally, we introduce new inter-grained and
cross-grained loss functions to further refine diverse multi-teacher knowledge
distillation transfer process, presenting distribution gaps and a harmonised
understanding of form documents. Through a comprehensive evaluation across
publicly available form document understanding datasets, our proposed model
consistently outperforms existing baselines, showcasing its efficacy in
handling the intricate structures and content of visually complex form
documents.