Towards Open-World Co-Salient Object Detection with Generative
  Uncertainty-aware Group Selective Exchange-Masking
Yang Wu Shenglong Hu Huihui Song Kaihua Zhang Bo Liu Dong Liu
The traditional definition of co-salient object detection (CoSOD) task is to
segment the common salient objects in a group of relevant images. This
definition is based on an assumption of group consensus consistency that is not
always reasonable in the open-world setting, which results in robustness issue
in the model when dealing with irrelevant images in the inputting image group
under the open-word scenarios. To tackle this problem, we introduce a group
selective exchange-masking (GSEM) approach for enhancing the robustness of the
CoSOD model. GSEM takes two groups of images as input, each containing
different types of salient objects. Based on the mixed metric we designed, GSEM
selects a subset of images from each group using a novel learning-based
strategy, then the selected images are exchanged. To simultaneously consider
the uncertainty introduced by irrelevant images and the consensus features of
the remaining relevant images in the group, we designed a latent variable
generator branch and CoSOD transformer branch. The former is composed of a
vector quantised-variational autoencoder to generate stochastic global
variables that model uncertainty. The latter is designed to capture
correlation-based local features that include group consensus. Finally, the
outputs of the two branches are merged and passed to a transformer-based
decoder to generate robust predictions. Taking into account that there are
currently no benchmark datasets specifically designed for open-world scenarios,
we constructed three open-world benchmark datasets, namely OWCoSal, OWCoSOD,
and OWCoCA, based on existing datasets. By breaking the group-consistency
assumption, these datasets provide effective simulations of real-world
scenarios and can better evaluate the robustness and practicality of models.