Isotropy, Clusters, and Classifiers
Timothee Mickus Stig-Arne Grönroos Joseph Attieh
Whether embedding spaces use all their dimensions equally, i.e., whether they
are isotropic, has been a recent subject of discussion. Evidence has been
accrued both for and against enforcing isotropy in embedding spaces. In the
present paper, we stress that isotropy imposes requirements on the embedding
space that are not compatible with the presence of clusters -- which also
negatively impacts linear classification objectives. We demonstrate this fact
empirically and use it to shed light on previous results from the literature.