A Regularization-based Transfer Learning Method for Information
  Extraction via Instructed Graph Decoder
Kedi Chen Jie Zhou Qin Chen Shunyu Liu Liang He
Information extraction (IE) aims to extract complex structured information
from the text. Numerous datasets have been constructed for various IE tasks,
leading to time-consuming and labor-intensive data annotations. Nevertheless,
most prevailing methods focus on training task-specific models, while the
common knowledge among different IE tasks is not explicitly modeled. Moreover,
the same phrase may have inconsistent labels in different tasks, which poses a
big challenge for knowledge transfer using a unified model. In this study, we
propose a regularization-based transfer learning method for IE (TIE) via an
instructed graph decoder. Specifically, we first construct an instruction pool
for datasets from all well-known IE tasks, and then present an instructed graph
decoder, which decodes various complex structures into a graph uniformly based
on corresponding instructions. In this way, the common knowledge shared with
existing datasets can be learned and transferred to a new dataset with new
labels. Furthermore, to alleviate the label inconsistency problem among various
IE tasks, we introduce a task-specific regularization strategy, which does not
update the gradients of two tasks with 'opposite direction'. We conduct
extensive experiments on 12 datasets spanning four IE tasks, and the results
demonstrate the great advantages of our proposed method