Supervised Manifold Learning via Random Forest Geometry-Preserving
  Proximities
Jake S. Rhodes
Manifold learning approaches seek the intrinsic, low-dimensional data
structure within a high-dimensional space. Mainstream manifold learning
algorithms, such as Isomap, UMAP, $t$-SNE, Diffusion Map, and Laplacian
Eigenmaps do not use data labels and are thus considered unsupervised. Existing
supervised extensions of these methods are limited to classification problems
and fall short of uncovering meaningful embeddings due to their construction
using order non-preserving, class-conditional distances. In this paper, we show
the weaknesses of class-conditional manifold learning quantitatively and
visually and propose an alternate choice of kernel for supervised
dimensionality reduction using a data-geometry-preserving variant of random
forest proximities as an initialization for manifold learning methods. We show
that local structure preservation using these proximities is near universal
across manifold learning approaches and global structure is properly maintained
using diffusion-based algorithms.