Representation Learning via Variational Bayesian Networks
Oren Barkan Avi Caciularu Idan Rejwan Ori Katz Jonathan Weill Itzik Malkiel Noam Koenigstein
We present Variational Bayesian Network (VBN) - a novel Bayesian entity
representation learning model that utilizes hierarchical and relational side
information and is particularly useful for modeling entities in the
``long-tail'', where the data is scarce. VBN provides better modeling for
long-tail entities via two complementary mechanisms: First, VBN employs
informative hierarchical priors that enable information propagation between
entities sharing common ancestors. Additionally, VBN models explicit relations
between entities that enforce complementary structure and consistency, guiding
the learned representations towards a more meaningful arrangement in space.
Second, VBN represents entities by densities (rather than vectors), hence
modeling uncertainty that plays a complementary role in coping with data
scarcity. Finally, we propose a scalable Variational Bayes optimization
algorithm that enables fast approximate Bayesian inference. We evaluate the
effectiveness of VBN on linguistic, recommendations, and medical inference
tasks. Our findings show that VBN outperforms other existing methods across
multiple datasets, and especially in the long-tail.