Spatial Structure Constraints for Weakly Supervised Semantic
  Segmentation
Tao Chen Yazhou Yao Xingguo Huang Zechao Li Liqiang Nie Jinhui Tang
The image-level label has prevailed in weakly supervised semantic
segmentation tasks due to its easy availability. Since image-level labels can
only indicate the existence or absence of specific categories of objects,
visualization-based techniques have been widely adopted to provide object
location clues. Considering class activation maps (CAMs) can only locate the
most discriminative part of objects, recent approaches usually adopt an
expansion strategy to enlarge the activation area for more integral object
localization. However, without proper constraints, the expanded activation will
easily intrude into the background region. In this paper, we propose spatial
structure constraints (SSC) for weakly supervised semantic segmentation to
alleviate the unwanted object over-activation of attention expansion.
Specifically, we propose a CAM-driven reconstruction module to directly
reconstruct the input image from deep CAM features, which constrains the
diffusion of last-layer object attention by preserving the coarse spatial
structure of the image content. Moreover, we propose an activation
self-modulation module to refine CAMs with finer spatial structure details by
enhancing regional consistency. Without external saliency models to provide
background clues, our approach achieves 72.7\% and 47.0\% mIoU on the PASCAL
VOC 2012 and COCO datasets, respectively, demonstrating the superiority of our
proposed approach.