Semi-supervised segmentation of land cover images using nonlinear
  canonical correlation analysis with multiple features and t-SNE
Hong Wei James Xiao Yichao Zhang Xia Hong
Image segmentation is a clustering task whereby each pixel is assigned a
cluster label. Remote sensing data usually consists of multiple bands of
spectral images in which there exist semantically meaningful land cover
subregions, co-registered with other source data such as LIDAR (LIght Detection
And Ranging) data, where available. This suggests that, in order to account for
spatial correlation between pixels, a feature vector associated with each pixel
may be a vectorized tensor representing the multiple bands and a local patch as
appropriate. Similarly, multiple types of texture features based on a pixel's
local patch would also be beneficial for encoding locally statistical
information and spatial variations, without necessarily labelling pixel-wise a
large amount of ground truth, then training a supervised model, which is
sometimes impractical. In this work, by resorting to label only a small
quantity of pixels, a new semi-supervised segmentation approach is proposed.
Initially, over all pixels, an image data matrix is created in high dimensional
feature space. Then, t-SNE projects the high dimensional data onto 3D
embedding. By using radial basis functions as input features, which use the
labelled data samples as centres, to pair with the output class labels, a
modified canonical correlation analysis algorithm, referred to as RBF-CCA, is
introduced which learns the associated projection matrix via the small labelled
data set. The associated canonical variables, obtained for the full image, are
applied by k-means clustering algorithm. The proposed semi-supervised RBF-CCA
algorithm has been implemented on several remotely sensed multispectral images,
demonstrating excellent segmentation results.