Genetic Learning for Designing Sim-to-Real Data Augmentations
Bram Vanherle Nick Michiels Frank Van Reeth
Data augmentations are useful in closing the sim-to-real domain gap when
training on synthetic data. This is because they widen the training data
distribution, thus encouraging the model to generalize better to other domains.
Many image augmentation techniques exist, parametrized by different settings,
such as strength and probability. This leads to a large space of different
possible augmentation policies. Some policies work better than others for
overcoming the sim-to-real gap for specific datasets, and it is unclear why.
This paper presents two different interpretable metrics that can be combined to
predict how well a certain augmentation policy will work for a specific
sim-to-real setting, focusing on object detection. We validate our metrics by
training many models with different augmentation policies and showing a strong
correlation with performance on real data. Additionally, we introduce
GeneticAugment, a genetic programming method that can leverage these metrics to
automatically design an augmentation policy for a specific dataset without
needing to train a model.