Word Embedding with Neural Probabilistic Prior
Shaogang Ren Dingcheng Li Ping Li
To improve word representation learning, we propose a probabilistic prior
which can be seamlessly integrated with word embedding models. Different from
previous methods, word embedding is taken as a probabilistic generative model,
and it enables us to impose a prior regularizing word representation learning.
The proposed prior not only enhances the representation of embedding vectors
but also improves the model's robustness and stability. The structure of the
proposed prior is simple and effective, and it can be easily implemented and
flexibly plugged in most existing word embedding models. Extensive experiments
show the proposed method improves word representation on various tasks.