Grid Jigsaw Representation with CLIP: A New Perspective on Image
  Clustering
Zijie Song Zhenzhen Hu Richang Hong
Unsupervised representation learning for image clustering is essential in
computer vision. Although the advancement of visual models has improved image
clustering with efficient visual representations, challenges still remain.
Firstly, these features often lack the ability to represent the internal
structure of images, hindering the accurate clustering of visually similar
images. Secondly, the existing features tend to lack finer-grained semantic
labels, limiting the ability to capture nuanced differences and similarities
between images.
  In this paper, we first introduce Jigsaw based strategy method for image
clustering called Grid Jigsaw Representation (GJR) with systematic exposition
from pixel to feature in discrepancy against human and computer. We emphasize
that this algorithm, which mimics human jigsaw puzzle, can effectively improve
the model to distinguish the spatial feature between different samples and
enhance the clustering ability. GJR modules are appended to a variety of deep
convolutional networks and tested with significant improvements on a wide range
of benchmark datasets including CIFAR-10, CIFAR-100/20, STL-10, ImageNet-10 and
ImageNetDog-15.
  On the other hand, convergence efficiency is always an important challenge
for unsupervised image clustering. Recently, pretrained representation learning
has made great progress and released models can extract mature visual
representations. It is obvious that use the pretrained model as feature
extractor can speed up the convergence of clustering where our aim is to
provide new perspective in image clustering with reasonable resource
application and provide new baseline. Further, we innovate pretrain-based Grid
Jigsaw Representation (pGJR) with improvement by GJR. The experiment results
show the effectiveness on the clustering task with respect to the ACC, NMI and
ARI three metrics and super fast convergence speed.