OV-PARTS: Towards Open-Vocabulary Part Segmentation
Meng Wei Xiaoyu Yue Wenwei Zhang Shu Kong Xihui Liu Jiangmiao Pang
Segmenting and recognizing diverse object parts is a crucial ability in
applications spanning various computer vision and robotic tasks. While
significant progress has been made in object-level Open-Vocabulary Semantic
Segmentation (OVSS), i.e., segmenting objects with arbitrary text, the
corresponding part-level research poses additional challenges. Firstly, part
segmentation inherently involves intricate boundaries, while limited annotated
data compounds the challenge. Secondly, part segmentation introduces an open
granularity challenge due to the diverse and often ambiguous definitions of
parts in the open world. Furthermore, the large-scale vision and language
models, which play a key role in the open vocabulary setting, struggle to
recognize parts as effectively as objects. To comprehensively investigate and
tackle these challenges, we propose an Open-Vocabulary Part Segmentation
(OV-PARTS) benchmark. OV-PARTS includes refined versions of two publicly
available datasets: Pascal-Part-116 and ADE20K-Part-234. And it covers three
specific tasks: Generalized Zero-Shot Part Segmentation, Cross-Dataset Part
Segmentation, and Few-Shot Part Segmentation, providing insights into
analogical reasoning, open granularity and few-shot adapting abilities of
models. Moreover, we analyze and adapt two prevailing paradigms of existing
object-level OVSS methods for OV-PARTS. Extensive experimental analysis is
conducted to inspire future research in leveraging foundational models for
OV-PARTS. The code and dataset are available at
https://github.com/OpenRobotLab/OV_PARTS.