Towards Generalizable Referring Image Segmentation via Target Prompt and
  Visual Coherence
Yajie Liu Pu Ge Haoxiang Ma Shichao Fan Qingjie Liu Di Huang Yunhong Wang
Referring image segmentation (RIS) aims to segment objects in an image
conditioning on free-from text descriptions. Despite the overwhelming progress,
it still remains challenging for current approaches to perform well on cases
with various text expressions or with unseen visual entities, limiting its
further application. In this paper, we present a novel RIS approach, which
substantially improves the generalization ability by addressing the two
dilemmas mentioned above. Specially, to deal with unconstrained texts, we
propose to boost a given expression with an explicit and crucial prompt, which
complements the expression in a unified context, facilitating target capturing
in the presence of linguistic style changes. Furthermore, we introduce a
multi-modal fusion aggregation module with visual guidance from a powerful
pretrained model to leverage spatial relations and pixel coherences to handle
the incomplete target masks and false positive irregular clumps which often
appear on unseen visual entities. Extensive experiments are conducted in the
zero-shot cross-dataset settings and the proposed approach achieves consistent
gains compared to the state-of-the-art, e.g., 4.15\%, 5.45\%, and 4.64\% mIoU
increase on RefCOCO, RefCOCO+ and ReferIt respectively, demonstrating its
effectiveness. Additionally, the results on GraspNet-RIS show that our approach
also generalizes well to new scenarios with large domain shifts.