On the Equivalency, Substitutability, and Flexibility of Synthetic Data
Che-Jui Chang Danrui Li Seonghyeon Moon Mubbasir Kapadia
We study, from an empirical standpoint, the efficacy of synthetic data in
real-world scenarios. Leveraging synthetic data for training perception models
has become a key strategy embraced by the community due to its efficiency,
scalability, perfect annotations, and low costs. Despite proven advantages, few
studies put their stress on how to efficiently generate synthetic datasets to
solve real-world problems and to what extent synthetic data can reduce the
effort for real-world data collection. To answer the questions, we
systematically investigate several interesting properties of synthetic data --
the equivalency of synthetic data to real-world data, the substitutability of
synthetic data for real data, and the flexibility of synthetic data generators
to close up domain gaps. Leveraging the M3Act synthetic data generator, we
conduct experiments on DanceTrack and MOT17. Our results suggest that synthetic
data not only enhances model performance but also demonstrates substitutability
for real data, with 60% to 80% replacement without performance loss. In
addition, our study of the impact of synthetic data distributions on downstream
performance reveals the importance of flexible data generators in narrowing
domain gaps for improved model adaptability.