Axis Tour: Word Tour Determines the Order of Axes in ICA-transformed
  Embeddings
Hiroaki Yamagiwa Yusuke Takase Hidetoshi Shimodaira
Word embedding is one of the most important components in natural language
processing, but interpreting high-dimensional embeddings remains a challenging
problem. To address this problem, Independent Component Analysis (ICA) is
identified as an effective solution. ICA-transformed word embeddings reveal
interpretable semantic axes; however, the order of these axes are arbitrary. In
this study, we focus on this property and propose a novel method, Axis Tour,
which optimizes the order of the axes. Inspired by Word Tour, a one-dimensional
word embedding method, we aim to improve the clarity of the word embedding
space by maximizing the semantic continuity of the axes. Furthermore, we show
through experiments on downstream tasks that Axis Tour constructs better
low-dimensional embeddings compared to both PCA and ICA.