Learning Multiplex Embeddings on Text-rich Networks with One Text
  Encoder
Bowen Jin Wentao Zhang Yu Zhang Yu Meng Han Zhao Jiawei Han
In real-world scenarios, texts in a network are often linked by multiple
semantic relations (e.g., papers in an academic network are referenced by other
publications, written by the same author, or published in the same venue),
where text documents and their relations form a multiplex text-rich network.
Mainstream text representation learning methods use pretrained language models
(PLMs) to generate one embedding for each text unit, expecting that all types
of relations between texts can be captured by these single-view embeddings.
However, this presumption does not hold particularly in multiplex text-rich
networks. Along another line of work, multiplex graph neural networks (GNNs)
directly initialize node attributes as a feature vector for node representation
learning, but they cannot fully capture the semantics of the nodes' associated
texts. To bridge these gaps, we propose METERN, a new framework for learning
Multiplex Embeddings on TExt-Rich Networks. In contrast to existing methods,
METERN uses one text encoder to model the shared knowledge across relations and
leverages a small number of parameters per relation to derive relation-specific
representations. This allows the encoder to effectively capture the multiplex
structures in the network while also preserving parameter efficiency. We
conduct experiments on nine downstream tasks in five networks from both
academic and e-commerce domains, where METERN outperforms baselines
significantly and consistently. The code is available at
https://github.com/PeterGriffinJin/METERN-submit.