Reliability in Semantic Segmentation: Can We Use Synthetic Data?
Thibaut Loiseau Tuan-Hung Vu Mickael Chen Patrick Pérez Matthieu Cord
Assessing the reliability of perception models to covariate shifts and
out-of-distribution (OOD) detection is crucial for safety-critical applications
such as autonomous vehicles. By nature of the task, however, the relevant data
is difficult to collect and annotate. In this paper, we challenge cutting-edge
generative models to automatically synthesize data for assessing reliability in
semantic segmentation. By fine-tuning Stable Diffusion, we perform zero-shot
generation of synthetic data in OOD domains or inpainted with OOD objects.
Synthetic data is employed to provide an initial assessment of pretrained
segmenters, thereby offering insights into their performance when confronted
with real edge cases. Through extensive experiments, we demonstrate a high
correlation between the performance on synthetic data and the performance on
real OOD data, showing the validity approach. Furthermore, we illustrate how
synthetic data can be utilized to enhance the calibration and OOD detection
capabilities of segmenters.