In this paper we propose a Deep Autoencoder Mixture Clustering (DAMIC) algorithm. It is based on a mixture of deep autoencoders where each cluster is represented by an autoencoder. A clustering network transforms the data into another space and then selects one of the clusters. Next, the autoencoder associated with this cluster is used to reconstruct the data-point. The clustering algorithm jointly learns the nonlinear data representation and the set of autoencoders. The optimal clustering is found by minimizing the reconstruction loss of the mixture of autoencoder network. Unlike other deep clustering algorithms, no regularization term is needed to avoid data collapsing to a single point. Our experimental evaluations on image and text corpora show significant improvement over state-of-the-art methods. Effective automatic grouping of objects into clusters is one of the fundamental problems in machine learning and data analysis. In many approaches, the first step toward clustering a dataset is extracting a feature vector from each object. This reduces the problem to the aggregation of groups of vectors in a feature space. A commonly used clustering algorithm in this case is the k-means. Clustering high-dimensional datasets is, however, hard because inter-point distances become less informative in high-dimensional spaces. As a result, representation learning has been used to map the input data into a low-dimensional feature space. In recent years, motivated by the success of deep neural network in supervised learning, there are many attempts to apply unsupervised deep learning approaches for clustering. Most methods are focused on clustering over a low-dimensional feature space of an autoencoder or a variational autoencoder. Recent good overviews of deep clustering methods can be found in BID1 and BID14 .Using deep neural networks, it is possible to learn nonlinear mappings allowing to transform the data into more clustering-friendly representations. A deep version of k-means is based on learning a data representation and applying k-means in the embedded space. A straightforward implementation of the deep k-means algorithm would lead, however, to a trivial solution where the features are collapsed to a single point in the embedded space and the centroids are collapsed into a single entity. The objective function of most deep clustering algorithms is, therefore, composed of a clustering term computed in the embedded space and a regularization term in the form of reconstruction error to avoid data collapsing. Deep Embedded Clustering (DEC) BID16 ) is first pre-trained using an autoencoder reconstruction loss and then optimizes cluster centroids in the embedded space through a Kullback-Leibeler divergence loss. Deep Clustering network (DCN) BID17 ) is another autoencoder-based method that uses k-means for clustering. Similar to DEC, in the first phase, the network is pre-trained using the autoencoder reconstruction loss. In the second phase, in contrast to DEC, the network is jointly trained using a mathematical combination of the autoencoder reconstruction loss and the k-means clustering loss function. Thus, due to the fact that strict cluster assignments were used during the training (instead of probabilities such as in DEC) the method requires an alternation process between the network training and the cluster updates.In this paper we propose an algorithm to perform unsupervised clustering within the mixture-ofexperts framework BID8 ). Each cluster is represented by an autoencoder neuralnetwork and the clustering itself is performed in a low-dimensional embedded space by a softmax classification layer that directs the input data to the most suitable autoencoder. Unlike most deep clustering algorithms the proposed algorithm is deep in nature and not a deep variant of a classical clustering algorithm. The proposed algorithm does not suffer from the clustering collapsing problem and therefore there is no need for regularization terms that have to be tuned separately for each dataset. Note that parameter tuning in clustering is problematic since it is based, either explicitly or implicitly, on the data labels which are not supposed to be available in the clustering process. Another major difference of the proposed method from previous approaches is the learning method of the embedding latent space where the actual clustering operation is taking place. In most previous methods, the embedded space is controlled by an autoencoder. Thus, in order to gain a good reconstruction, it requires to encode into the embedded space information that can be entirely irrelevant to the clustering process. In contrast, in our algorithm no decoding is applied to the clustering embedded space and the only goal of the embedded space is to find a good organization of the data into separated clusters.We validate the method on standard real datasets including various document and image corpora. Evidently, visible improvement from the respective state-of-art is observed for all the tested datasets. The contribution of this paper is twofold: (i) a novel deep learning clustering method that unlike deep variants of k-means, does not require a tuned regularization term to avoid clustering collapsing to a single point; and (ii) state-of-the-art performance on standard datasets. In this study we presented a clustering technique which leverages the strength of deep neural network. Our technique has two major properties: first, unlike most previous methods, the clusters are represented by an autoencoder network instead of a single centroid vector in the embedded space. This enables a much richer representation of each cluster. Second, The algorithm does not cause a data collapsing problem. Hence, there is no need for regularization terms that have to be tuned for each dataset separately. Experiments on a variety of real datasets showed the improved performance of the proposed algorithm.
We propose a deep clustering method where instead of a centroid each cluster is represented by an autoencoder Presents deep clustering based on a mixture of autoencoders, where data points are allocated to a cluster based the representation error if the autoencoder network were used to represent it. A deep clustering approach that uses an autoencoder framework to learn a low-dimensional embedding of the data simultaneously while clustering data using a deep neural network. A deep clustering method which represents each cluster with different auto-encoders, works in an end-to-end manner, and also can be used to cluster new incoming data without redoing the whole clustering procedure.