CoC-GAN: Employing Context Cluster for Unveiling a New Pathway in Image
  Generation
Zihao Wang Yiming Huang Ziyu Zhou
Image generation tasks are traditionally undertaken using Convolutional
Neural Networks (CNN) or Transformer architectures for feature aggregating and
dispatching. Despite the frequent application of convolution and attention
structures, these structures are not fundamentally required to solve the
problem of instability and the lack of interpretability in image generation. In
this paper, we propose a unique image generation process premised on the
perspective of converting images into a set of point clouds. In other words, we
interpret an image as a set of points. As such, our methodology leverages
simple clustering methods named Context Clustering (CoC) to generate images
from unordered point sets, which defies the convention of using convolution or
attention mechanisms. Hence, we exclusively depend on this clustering
technique, combined with the multi-layer perceptron (MLP) in a generative
model. Furthermore, we implement the integration of a module termed the 'Point
Increaser' for the model. This module is just an MLP tasked with generating
additional points for clustering, which are subsequently integrated within the
paradigm of the Generative Adversarial Network (GAN). We introduce this model
with the novel structure as the Context Clustering Generative Adversarial
Network (CoC-GAN), which offers a distinctive viewpoint in the domain of
feature aggregating and dispatching. Empirical evaluations affirm that our
CoC-GAN, devoid of convolution and attention mechanisms, exhibits outstanding
performance. Its interpretability, endowed by the CoC module, also allows for
visualization in our experiments. The promising results underscore the
feasibility of our method and thus warrant future investigations of applying
Context Clustering to more novel and interpretable image generation.