SKU-Patch: Towards Efficient Instance Segmentation for Unseen Objects in
  Auto-Store
Biqi Yang Weiliang Tang Xiaojie Gao Xianzhi Li Yun-Hui Liu Chi-Wing Fu Pheng-Ann Heng
In large-scale storehouses, precise instance masks are crucial for robotic
bin picking but are challenging to obtain. Existing instance segmentation
methods typically rely on a tedious process of scene collection, mask
annotation, and network fine-tuning for every single Stock Keeping Unit (SKU).
This paper presents SKU-Patch, a new patch-guided instance segmentation
solution, leveraging only a few image patches for each incoming new SKU to
predict accurate and robust masks, without tedious manual effort and model
re-training. Technical-wise, we design a novel transformer-based network with
(i) a patch-image correlation encoder to capture multi-level image features
calibrated by patch information and (ii) a patch-aware transformer decoder with
parallel task heads to generate instance masks. Extensive experiments on four
storehouse benchmarks manifest that SKU-Patch is able to achieve the best
performance over the state-of-the-art methods. Also, SKU-Patch yields an
average of nearly 100% grasping success rate on more than 50 unseen SKUs in a
robot-aided auto-store logistic pipeline, showing its effectiveness and
practicality.