Compositional Fusion of Signals in Data Embedding
Zhijin Guo Zhaozhen Xu Martha Lewis Nello Cristianini
Embeddings in AI convert symbolic structures into fixed-dimensional vectors,
effectively fusing multiple signals. However, the nature of this fusion in
real-world data is often unclear. To address this, we introduce two methods:
(1) Correlation-based Fusion Detection, measuring correlation between known
attributes and embeddings, and (2) Additive Fusion Detection, viewing
embeddings as sums of individual vectors representing attributes.
  Applying these methods, word embeddings were found to combine semantic and
morphological signals. BERT sentence embeddings were decomposed into individual
word vectors of subject, verb and object. In the knowledge graph-based
recommender system, user embeddings, even without training on demographic data,
exhibited signals of demographics like age and gender.
  This study highlights that embeddings are fusions of multiple signals, from
Word2Vec components to demographic hints in graph embeddings.