Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs
  in Language Pretraining
Jingcong Liang Rong Ye Meng Han Qi Zhang Ruofei Lai Xinyu Zhang Zhao Cao Xuanjing Huang Zhongyu Wei
The knowledge graph is a structure to store and represent knowledge, and
recent studies have discussed its capability to assist language models for
various applications. Some variations of knowledge graphs aim to record
arguments and their relations for computational argumentation tasks. However,
many must simplify semantic types to fit specific schemas, thus losing
flexibility and expression ability. In this paper, we propose the Hierarchical
Argumentation Graph (Hi-ArG), a new structure to organize arguments. We also
introduce two approaches to exploit Hi-ArG, including a text-graph multi-modal
model GreaseArG and a new pre-training framework augmented with graph
information. Experiments on two argumentation tasks have shown that after
further pre-training and fine-tuning, GreaseArG supersedes same-scale language
models on these tasks, while incorporating graph information during further
pre-training can also improve the performance of vanilla language models. Code
for this paper is available at https://github.com/ljcleo/Hi-ArG .