Diffusion Dataset Generation: Towards Closing the Sim2Real Gap for
  Pedestrian Detection
Andrew Farley Mohsen Zand Michael Greenspan
We propose a method that augments a simulated dataset using diffusion models
to improve the performance of pedestrian detection in real-world data. The high
cost of collecting and annotating data in the real-world has motivated the use
of simulation platforms to create training datasets. While simulated data is
inexpensive to collect and annotate, it unfortunately does not always closely
match the distribution of real-world data, which is known as the sim2real gap.
In this paper we propose a novel method of synthetic data creation meant to
close the sim2real gap for the challenging pedestrian detection task. Our
method uses a diffusion-based architecture to learn a real-world distribution
which, once trained, is used to generate datasets. We mix this generated data
with simulated data as a form of augmentation and show that training on a
combination of generated and simulated data increases average precision by as
much as 27.3% for pedestrian detection models in real-world data, compared
against training on purely simulated data.