Unexplainable Explanations: Towards Interpreting tSNE and UMAP
  Embeddings
Andrew Draganov Simon Dohn
It has become standard to explain neural network latent spaces with
attraction/repulsion dimensionality reduction (ARDR) methods like tSNE and
UMAP. This relies on the premise that structure in the 2D representation is
consistent with the structure in the model's latent space. However, this is an
unproven assumption -- we are unaware of any convergence guarantees for ARDR
algorithms. We work on closing this question by relating ARDR methods to
classical dimensionality reduction techniques. Specifically, we show that one
can fully recover a PCA embedding by applying attractions and repulsions onto a
randomly initialized dataset. We also show that, with a small change, Locally
Linear Embeddings (LLE) can reproduce ARDR embeddings. Finally, we formalize a
series of conjectures that, if true, would allow one to attribute structure in
the 2D embedding back to the input distribution.