Semi-DETR: Semi-Supervised Object Detection with Detection Transformers
Jiacheng Zhang Xiangru Lin Wei Zhang Kuo Wang Xiao Tan Junyu Han Errui Ding Jingdong Wang Guanbin Li
We analyze the DETR-based framework on semi-supervised object detection
(SSOD) and observe that (1) the one-to-one assignment strategy generates
incorrect matching when the pseudo ground-truth bounding box is inaccurate,
leading to training inefficiency; (2) DETR-based detectors lack deterministic
correspondence between the input query and its prediction output, which hinders
the applicability of the consistency-based regularization widely used in
current SSOD methods. We present Semi-DETR, the first transformer-based
end-to-end semi-supervised object detector, to tackle these problems.
Specifically, we propose a Stage-wise Hybrid Matching strategy that combines
the one-to-many assignment and one-to-one assignment strategies to improve the
training efficiency of the first stage and thus provide high-quality pseudo
labels for the training of the second stage. Besides, we introduce a Crossview
Query Consistency method to learn the semantic feature invariance of object
queries from different views while avoiding the need to find deterministic
query correspondence. Furthermore, we propose a Cost-based Pseudo Label Mining
module to dynamically mine more pseudo boxes based on the matching cost of
pseudo ground truth bounding boxes for consistency training. Extensive
experiments on all SSOD settings of both COCO and Pascal VOC benchmark datasets
show that our Semi-DETR method outperforms all state-of-the-art methods by
clear margins. The PaddlePaddle version code1 is at
https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/semi_det/semi_detr.