Semi-supervised Open-World Object Detection
Sahal Shaji Mullappilly Abhishek Singh Gehlot Rao Muhammad Anwer Fahad Shahbaz Khan Hisham Cholakkal
Conventional open-world object detection (OWOD) problem setting first
distinguishes known and unknown classes and then later incrementally learns the
unknown objects when introduced with labels in the subsequent tasks. However,
the current OWOD formulation heavily relies on the external human oracle for
knowledge input during the incremental learning stages. Such reliance on
run-time makes this formulation less realistic in a real-world deployment. To
address this, we introduce a more realistic formulation, named semi-supervised
open-world detection (SS-OWOD), that reduces the annotation cost by casting the
incremental learning stages of OWOD in a semi-supervised manner. We demonstrate
that the performance of the state-of-the-art OWOD detector dramatically
deteriorates in the proposed SS-OWOD setting. Therefore, we introduce a novel
SS-OWOD detector, named SS-OWFormer, that utilizes a feature-alignment scheme
to better align the object query representations between the original and
augmented images to leverage the large unlabeled and few labeled data. We
further introduce a pseudo-labeling scheme for unknown detection that exploits
the inherent capability of decoder object queries to capture object-specific
information. We demonstrate the effectiveness of our SS-OWOD problem setting
and approach for remote sensing object detection, proposing carefully curated
splits and baseline performance evaluations. Our experiments on 4 datasets
including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of
our approach. Our source code, models and splits are available here -
https://github.com/sahalshajim/SS-OWFormer