Object Detector Differences when using Synthetic and Real Training Data
Martin Georg Ljungqvist Otto Nordander Markus Skans Arvid Mildner Tony Liu Pierre Nugues
To train well-performing generalizing neural networks, sufficiently large and
diverse datasets are needed. Collecting data while adhering to privacy
legislation becomes increasingly difficult and annotating these large datasets
is both a resource-heavy and time-consuming task. An approach to overcome these
difficulties is to use synthetic data since it is inherently scalable and can
be automatically annotated. However, how training on synthetic data affects the
layers of a neural network is still unclear. In this paper, we train the YOLOv3
object detector on real and synthetic images from city environments. We perform
a similarity analysis using Centered Kernel Alignment (CKA) to explore the
effects of training on synthetic data on a layer-wise basis. The analysis
captures the architecture of the detector while showing both different and
similar patterns between different models. With this similarity analysis we
want to give insights on how training synthetic data affects each layer and to
give a better understanding of the inner workings of complex neural networks.
The results show that the largest similarity between a detector trained on real
data and a detector trained on synthetic data was in the early layers, and the
largest difference was in the head part. The results also show that no major
difference in performance or similarity could be seen between frozen and
unfrozen backbone.