Towards Open-World Segmentation of Parts
Tai-Yu Pan Qing Liu Wei-Lun Chao Brian Price
Segmenting object parts such as cup handles and animal bodies is important in
many real-world applications but requires more annotation effort. The largest
dataset nowadays contains merely two hundred object categories, implying the
difficulty to scale up part segmentation to an unconstrained setting. To
address this, we propose to explore a seemingly simplified but empirically
useful and scalable task, class-agnostic part segmentation. In this problem, we
disregard the part class labels in training and instead treat all of them as a
single part class. We argue and demonstrate that models trained without part
classes can better localize parts and segment them on objects unseen in
training. We then present two further improvements. First, we propose to make
the model object-aware, leveraging the fact that parts are "compositions",
whose extents are bounded by the corresponding objects and whose appearances
are by nature not independent but bundled. Second, we introduce a novel
approach to improve part segmentation on unseen objects, inspired by an
interesting finding -- for unseen objects, the pixel-wise features extracted by
the model often reveal high-quality part segments. To this end, we propose a
novel self-supervised procedure that iterates between pixel clustering and
supervised contrastive learning that pulls pixels closer or pushes them away.
Via extensive experiments on PartImageNet and Pascal-Part, we show notable and
consistent gains by our approach, essentially a critical step towards
open-world part segmentation.