PM-DETR: Domain Adaptive Prompt Memory for Object Detection with
  Transformers
Peidong Jia Jiaming Liu Senqiao Yang Jiarui Wu Xiaodong Xie Shanghang Zhang
The Transformer-based detectors (i.e., DETR) have demonstrated impressive
performance on end-to-end object detection. However, transferring DETR to
different data distributions may lead to a significant performance degradation.
Existing adaptation techniques focus on model-based approaches, which aim to
leverage feature alignment to narrow the distribution shift between different
domains. In this study, we propose a hierarchical Prompt Domain Memory (PDM)
for adapting detection transformers to different distributions. PDM
comprehensively leverages the prompt memory to extract domain-specific
knowledge and explicitly constructs a long-term memory space for the data
distribution, which represents better domain diversity compared to existing
methods. Specifically, each prompt and its corresponding distribution value are
paired in the memory space, and we inject top M distribution-similar prompts
into the input and multi-level embeddings of DETR. Additionally, we introduce
the Prompt Memory Alignment (PMA) to reduce the discrepancy between the source
and target domains by fully leveraging the domain-specific knowledge extracted
from the prompt domain memory. Extensive experiments demonstrate that our
method outperforms state-of-the-art domain adaptive object detection methods on
three benchmarks, including scene, synthetic to real, and weather adaptation.
Codes will be released.