Low-Resource Multi-Granularity Academic Function Recognition Based on
  Multiple Prompt Knowledge
Jiawei Liu Zi Xiong Yi Jiang Yongqiang Ma Wei Lu Yong Huang Qikai Cheng
Fine-tuning pre-trained language models (PLMs), e.g., SciBERT, generally
requires large numbers of annotated data to achieve state-of-the-art
performance on a range of NLP tasks in the scientific domain. However,
obtaining the fine-tune data for scientific NLP task is still challenging and
expensive. Inspired by recent advancement in prompt learning, in this paper, we
propose the Mix Prompt Tuning (MPT), which is a semi-supervised method to
alleviate the dependence on annotated data and improve the performance of
multi-granularity academic function recognition tasks with a small number of
labeled examples. Specifically, the proposed method provides multi-perspective
representations by combining manual prompt templates with automatically learned
continuous prompt templates to help the given academic function recognition
task take full advantage of knowledge in PLMs. Based on these prompt templates
and the fine-tuned PLM, a large number of pseudo labels are assigned to the
unlabeled examples. Finally, we fine-tune the PLM using the pseudo training
set. We evaluate our method on three academic function recognition tasks of
different granularity including the citation function, the abstract sentence
function, and the keyword function, with datasets from computer science domain
and biomedical domain. Extensive experiments demonstrate the effectiveness of
our method and statistically significant improvements against strong baselines.
In particular, it achieves an average increase of 5% in Macro-F1 score compared
with fine-tuning, and 6% in Macro-F1 score compared with other semi-supervised
method under low-resource settings. In addition, MPT is a general method that
can be easily applied to other low-resource scientific classification tasks.