HILL: Hierarchy-aware Information Lossless Contrastive Learning for
  Hierarchical Text Classification
He Zhu Junran Wu Ruomei Liu Yue Hou Ze Yuan Shangzhe Li Yicheng Pan Ke Xu
Existing self-supervised methods in natural language processing (NLP),
especially hierarchical text classification (HTC), mainly focus on
self-supervised contrastive learning, extremely relying on human-designed
augmentation rules to generate contrastive samples, which can potentially
corrupt or distort the original information. In this paper, we tend to
investigate the feasibility of a contrastive learning scheme in which the
semantic and syntactic information inherent in the input sample is adequately
reserved in the contrastive samples and fused during the learning process.
Specifically, we propose an information lossless contrastive learning strategy
for HTC, namely \textbf{H}ierarchy-aware \textbf{I}nformation \textbf{L}ossless
contrastive \textbf{L}earning (HILL), which consists of a text encoder
representing the input document, and a structure encoder directly generating
the positive sample. The structure encoder takes the document embedding as
input, extracts the essential syntactic information inherent in the label
hierarchy with the principle of structural entropy minimization, and injects
the syntactic information into the text representation via hierarchical
representation learning. Experiments on three common datasets are conducted to
verify the superiority of HILL.