Description-Enhanced Label Embedding Contrastive Learning for Text
  Classification
Kun Zhang Le Wu Guangyi Lv Enhong Chen Shulan Ruan Jing Liu Zhiqiang Zhang Jun Zhou Meng Wang
Text Classification is one of the fundamental tasks in natural language
processing, which requires an agent to determine the most appropriate category
for input sentences. Recently, deep neural networks have achieved impressive
performance in this area, especially Pre-trained Language Models (PLMs).
Usually, these methods concentrate on input sentences and corresponding
semantic embedding generation. However, for another essential component:
labels, most existing works either treat them as meaningless one-hot vectors or
use vanilla embedding methods to learn label representations along with model
training, underestimating the semantic information and guidance that these
labels reveal. To alleviate this problem and better exploit label information,
in this paper, we employ Self-Supervised Learning (SSL) in model learning
process and design a novel self-supervised Relation of Relation (R2)
classification task for label utilization from a one-hot manner perspective.
Then, we propose a novel Relation of Relation Learning Network (R2-Net) for
text classification, in which text classification and R2 classification are
treated as optimization targets. Meanwhile, triplet loss is employed to enhance
the analysis of differences and connections among labels. Moreover, considering
that one-hot usage is still short of exploiting label information, we
incorporate external knowledge from WordNet to obtain multi-aspect descriptions
for label semantic learning and extend R2-Net to a novel Description-Enhanced
Label Embedding network (DELE) from a label embedding perspective. ...