Ablation Study to Clarify the Mechanism of Object Segmentation in
  Multi-Object Representation Learning
Takayuki Komatsu Yoshiyuki Ohmura Yasuo Kuniyoshi
Multi-object representation learning aims to represent complex real-world
visual input using the composition of multiple objects. Representation learning
methods have often used unsupervised learning to segment an input image into
individual objects and encode these objects into each latent vector. However,
it is not clear how previous methods have achieved the appropriate segmentation
of individual objects. Additionally, most of the previous methods regularize
the latent vectors using a Variational Autoencoder (VAE). Therefore, it is not
clear whether VAE regularization contributes to appropriate object
segmentation. To elucidate the mechanism of object segmentation in multi-object
representation learning, we conducted an ablation study on MONet, which is a
typical method. MONet represents multiple objects using pairs that consist of
an attention mask and the latent vector corresponding to the attention mask.
Each latent vector is encoded from the input image and attention mask. Then,
the component image and attention mask are decoded from each latent vector. The
loss function of MONet consists of 1) the sum of reconstruction losses between
the input image and decoded component image, 2) the VAE regularization loss of
the latent vector, and 3) the reconstruction loss of the attention mask to
explicitly encode shape information. We conducted an ablation study on these
three loss functions to investigate the effect on segmentation performance. Our
results showed that the VAE regularization loss did not affect segmentation
performance and the others losses did affect it. Based on this result, we
hypothesize that it is important to maximize the attention mask of the image
region best represented by a single latent vector corresponding to the
attention mask. We confirmed this hypothesis by evaluating a new loss function
with the same mechanism as the hypothesis.