DiffusionEngine: Diffusion Model is Scalable Data Engine for Object
  Detection
Manlin Zhang Jie Wu Yuxi Ren Ming Li Jie Qin Xuefeng Xiao Wei Liu Rui Wang Min Zheng Andy J. Ma
Data is the cornerstone of deep learning. This paper reveals that the
recently developed Diffusion Model is a scalable data engine for object
detection. Existing methods for scaling up detection-oriented data often
require manual collection or generative models to obtain target images,
followed by data augmentation and labeling to produce training pairs, which are
costly, complex, or lacking diversity. To address these issues, we
presentDiffusionEngine (DE), a data scaling-up engine that provides
high-quality detection-oriented training pairs in a single stage. DE consists
of a pre-trained diffusion model and an effective Detection-Adapter,
contributing to generating scalable, diverse and generalizable detection data
in a plug-and-play manner. Detection-Adapter is learned to align the implicit
semantic and location knowledge in off-the-shelf diffusion models with
detection-aware signals to make better bounding-box predictions. Additionally,
we contribute two datasets, i.e., COCO-DE and VOC-DE, to scale up existing
detection benchmarks for facilitating follow-up research. Extensive experiments
demonstrate that data scaling-up via DE can achieve significant improvements in
diverse scenarios, such as various detection algorithms, self-supervised
pre-training, data-sparse, label-scarce, cross-domain, and semi-supervised
learning. For example, when using DE with a DINO-based adapter to scale up
data, mAP is improved by 3.1% on COCO, 7.6% on VOC, and 11.5% on Clipart.