Why current rain denoising models fail on CycleGAN created rain images
  in autonomous driving
Michael Kranl Hubert Ramsauer Bernhard Knapp
One of the main tasks of an autonomous agent in a vehicle is to correctly
perceive its environment. Much of the data that needs to be processed is
collected by optical sensors such as cameras. Unfortunately, the data collected
in this way can be affected by a variety of factors, including environmental
influences such as inclement weather conditions (e.g., rain). Such noisy data
can cause autonomous agents to take wrong decisions with potentially fatal
outcomes. This paper addresses the rain image challenge by two steps: First,
rain is artificially added to a set of clear-weather condition images using a
Generative Adversarial Network (GAN). This yields good/bad weather image pairs
for training de-raining models. This artificial generation of rain images is
sufficiently realistic as in 7 out of 10 cases, human test subjects believed
the generated rain images to be real. In a second step, this paired good/bad
weather image data is used to train two rain denoising models, one based
primarily on a Convolutional Neural Network (CNN) and the other using a Vision
Transformer. This rain de-noising step showed limited performance as the
quality gain was only about 15%. This lack of performance on realistic rain
images as used in our study is likely due to current rain de-noising models
being developed for simplistic rain overlay data. Our study shows that there is
ample space for improvement of de-raining models in autonomous driving.