Hierarchical Knowledge Distillation on Text Graph for Data-limited
  Attribute Inference
Quan Li Shixiong Jing Lingwei Chen
The popularization of social media increases user engagements and generates a
large amount of user-oriented data. Among them, text data (e.g., tweets, blogs)
significantly attracts researchers and speculators to infer user attributes
(e.g., age, gender, location) for fulfilling their intents. Generally, this
line of work casts attribute inference as a text classification problem, and
starts to leverage graph neural networks (GNNs) to utilize higher-level
representations of source texts. However, these text graphs are constructed
over words, suffering from high memory consumption and ineffectiveness on few
labeled texts. To address this challenge, we design a text-graph-based few-shot
learning model for attribute inferences on social media text data. Our model
first constructs and refines a text graph using manifold learning and message
passing, which offers a better trade-off between expressiveness and complexity.
Afterwards, to further use cross-domain texts and unlabeled texts to improve
few-shot performance, a hierarchical knowledge distillation is devised over
text graph to optimize the problem, which derives better text representations,
and advances model generalization ability. Experiments on social media datasets
demonstrate the state-of-the-art performance of our model on attribute
inferences with considerably fewer labeled texts.