Classification and clustering have been studied separately in machine learning and computer vision. Inspired by the recent success of deep learning models in solving various vision problems (e.g., object recognition, semantic segmentation) and the fact that humans serve as the gold standard in assessing clustering algorithms, here, we advocate for a unified treatment of the two problems and suggest that hierarchical frameworks that progressively build complex patterns on top of the simpler ones (e.g., convolutional neural networks) offer a promising solution. We do not dwell much on the learning mechanisms in these frameworks as they are still a matter of debate, with respect to biological constraints. Instead, we emphasize on the compositionality of the real world structures and objects. In particular, we show that CNNs, trained end to end using back propagation with noisy labels, are able to cluster data points belonging to several overlapping shapes, and do so much better than the state of the art algorithms. The main takeaway lesson from our study is that mechanisms of human vision, particularly the hierarchal organization of the visual ventral stream should be taken into account in clustering algorithms (e.g., for learning representations in an unsupervised manner or with minimum supervision) to reach human level clustering performance. This, by no means, suggests that other methods do not hold merits. For example, methods relying on pairwise affinities (e.g., spectral clustering) have been very successful in many cases but still fail in some cases (e.g., overlapping clusters). Clustering, a.k.a unsupervised classification or nonparametric density estimation, is central to many data-driven domains and has been studied heavily in the past. The task in clustering is to group a given collection of unlabeled patterns into meaningful clusters such that objects within a cluster are more similar to each other than they are to objects in other clusters. Clustering provides a summary representation of data at a coarse level and is used widely in many disciplines (e.g., computer version, bioinformatics, text processing) for exploratory data analysis (a.k.a pattern mining) as well as representation learning (e.g., bag of words). Despite the introduction of thousands of clustering algorithms in the past BID0 , some challenges still remain. For instance, existing algorithms fall short in dealing with different cluster shapes, high dimensions, automatically determining the number of clusters or other parameters, large amounts of data, choosing the appropriate similarity measure, incorporating domain knowledge, and cluster evaluation. Further, no clustering algorithm can consistently win over other algorithms, handle all test cases, and perform at the level of humans.Deep neural networks have become a dominant approach to solve various tasks across many fields. They have been proven successful in several domains including computer vision BID16 , natural language processing BID6 , and speech recognition BID8 for tasks such as scene and object classification BID16 , pixel-level labeling for image segmentation BID21 ; BID34 , modeling attention ; , image generation BID11 , robot arm control BID19 , speech recognition BID12 , playing Atari games BID23 and beating the Go champion. BID28 adopted in this work.Deep Convolutional Neural Networks (CNNs) BID17 have been particularly successful over vision problems. One reason is that nearby pixels in natural scenes are highly correlated. Further natural objects are compositional. These facts allow applying the same filters across spatial locations (and hence share weights), and build complex filters from simpler ones to detect highlevel patterns (e.g., object parts, objects). We advocate that these properties are highly appealing when dealing with clustering problems. For instance, the classic two half moons example can be solved by applying a filter that is selective to each half moon. Or, when two clusters with different shapes overlap, the problem can be solved by having filters responding to each shape. Solving these cases is very challenging by just looking at local regions around points and being blind to the high-level patterns. Incorporating domain knowledge, while working in some cases, does not give a general solution for solving all clustering problems. The human visual system easily solves these 2D problems because it is a general system with a rich set of learned or evolved filters. We believe that deep CNNs, although imperfect models of the human vision as they lack feedback and lateral connections carry a huge promise for solving clustering tasks. Further, as we will argue, they offer a unified solution to both classification and clustering tasks.The current demarcation between classification and clustering becomes murky when we notice that researchers often refer to human judgments in evaluating the outcomes of clustering algorithms. Indeed, humans learn quite a lot about the visual world during their life time. Moreover, the structure of the visual system has been fine-tuned through the evolution. Thus, certainly, there is a learning component involved which has been often neglected in formulating clustering algorithms. While this is sensible from an application point of view (e.g., pattern mining), not only it limits the pursuit for stronger algorithms but also narrows our understanding of human vision.Learning techniques have been utilized for clustering in the past (e.g., BID1 ; BID25 ), for example for tuning parameters (e.g., BID1 ). Deep networks have also been exploited for clustering (e.g., BID14 ; BID13 BID32 ). However, to our knowledge, while CNNs have been already adopted for image segmentation, so far they have not been exploited for generic clustering. Our goal is to investigate such possibility. To this end, instead of borrowing from clustering to do image segmentation, we follow the opposite direction and propose a deep learning based approach to clustering.Our method builds on the fully convolutional network literature, in particular, recent work on edge detection and semantic segmentation which utilize multi-scale local and non-local cues BID28 . Thanks to a high volume of labeled data, high capacity of deep networks, powerful optimization algorithms, and high computational power, deep models win on these tasks. We are also strongly inspired by the works showing the high resemblance between human vision mechanisms and CNNs from behavioral, electrophysiological, and computational aspects (e.g., BID33 BID9 ; BID17 ; BID16 ; BID3 . Our study enriches our understanding of the concept of clustering and its relation to classification. We argued that deep neural networks, especially CNNs, hold a great promise for data clustering. We are motivated by the fact that human vision (and learning) is a general system capable of solving both classification and clustering tasks thus blurring the current dichotomy in treating these problems. Our results show that CNNs can successfully handle complex and occluded clusters much better than other algorithms. This means that a learning mechanism, unsupervised or with minimal supervision, seems inevitable in capturing complex cluster shapes.While our formulation is supervised, feeding the labels to the network is not always consistent. This is where our work differs from semantic segmentation and instance level segmentation. We exploited the mean squared loss to train the network. It might be possible to define other loss functions to teach the network more efficiency using less number of training data or even with weaker labels. One possibility is the pairwise accuracy that we used here for evaluation. Instead of correctly classifying labels, the emphasis can be placed on correctly predicting whether two points belong to the same cluster, regardless of cluster identities (i.e., class labels may vary).Notice that while here we focused on synthetic stimuli, variations of the proposed CNN architecture, have been successfully applied to natural image segmentation. Thus, CNNs offer a unified solution that can be applied to different data modalities and even to higher dimensional data. Further work is needed to extend this line of work to higher dimensions, and more versatile types of cluster shapes (e.g., free form curves, Gestalt examples, density-based clusters). In this regard, adopting CNNs trained on natural images containing a rich set of intermediate-and high-level patterns can give invaluable insights.
Human-like Clustering with CNNs The paper validates the idea that deep convolutional neural networks could learn to cluster input data better than other clustering methods by noting their ability to interpret the context of every input point due to a large field of view. This work combines deep learning for feature representation with the task of human-like unsupervised grouping.