Teach model to answer questions after comprehending the document
Ruiqing Sun Ping Jian
Multi-choice Machine Reading Comprehension (MRC) is a challenging extension
of Natural Language Processing (NLP) that requires the ability to comprehend
the semantics and logical relationships between entities in a given text. The
MRC task has traditionally been viewed as a process of answering questions
based on the given text. This single-stage approach has often led the network
to concentrate on generating the correct answer, potentially neglecting the
comprehension of the text itself. As a result, many prevalent models have faced
challenges in performing well on this task when dealing with longer texts. In
this paper, we propose a two-stage knowledge distillation method that teaches
the model to better comprehend the document by dividing the MRC task into two
separate stages. Our experimental results show that the student model, when
equipped with our method, achieves significant improvements, demonstrating the
effectiveness of our method.