Meta-ZSDETR: Zero-shot DETR with Meta-learning
Lu Zhang Chenbo Zhang Jiajia Zhao Jihong Guan Shuigeng Zhou
Zero-shot object detection aims to localize and recognize objects of unseen
classes. Most of existing works face two problems: the low recall of RPN in
unseen classes and the confusion of unseen classes with background. In this
paper, we present the first method that combines DETR and meta-learning to
perform zero-shot object detection, named Meta-ZSDETR, where model training is
formalized as an individual episode based meta-learning task. Different from
Faster R-CNN based methods that firstly generate class-agnostic proposals, and
then classify them with visual-semantic alignment module, Meta-ZSDETR directly
predict class-specific boxes with class-specific queries and further filter
them with the predicted accuracy from classification head. The model is
optimized with meta-contrastive learning, which contains a regression head to
generate the coordinates of class-specific boxes, a classification head to
predict the accuracy of generated boxes, and a contrastive head that utilizes
the proposed contrastive-reconstruction loss to further separate different
classes in visual space. We conduct extensive experiments on two benchmark
datasets MS COCO and PASCAL VOC. Experimental results show that our method
outperforms the existing ZSD methods by a large margin.