SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete
  Diffusion Process
Mengyu Wang Henghui Ding Jun Hao Liew Jiajun Liu Yao Zhao Yunchao Wei
In this paper, we explore a principal way to enhance the quality of object
masks produced by different segmentation models. We propose a model-agnostic
solution called SegRefiner, which offers a novel perspective on this problem by
interpreting segmentation refinement as a data generation process. As a result,
the refinement process can be smoothly implemented through a series of
denoising diffusion steps. Specifically, SegRefiner takes coarse masks as
inputs and refines them using a discrete diffusion process. By predicting the
label and corresponding states-transition probabilities for each pixel,
SegRefiner progressively refines the noisy masks in a conditional denoising
manner. To assess the effectiveness of SegRefiner, we conduct comprehensive
experiments on various segmentation tasks, including semantic segmentation,
instance segmentation, and dichotomous image segmentation. The results
demonstrate the superiority of our SegRefiner from multiple aspects. Firstly,
it consistently improves both the segmentation metrics and boundary metrics
across different types of coarse masks. Secondly, it outperforms previous
model-agnostic refinement methods by a significant margin. Lastly, it exhibits
a strong capability to capture extremely fine details when refining
high-resolution images. The source code and trained models are available at
https://github.com/MengyuWang826/SegRefiner.