Improving Panoptic Segmentation for Nighttime or Low-Illumination Urban
  Driving Scenes
Ankur Chrungoo
Autonomous vehicles and driving systems use scene parsing as an essential
tool to understand the surrounding environment. Panoptic segmentation is a
state-of-the-art technique which proves to be pivotal in this use case. Deep
learning-based architectures have been utilized for effective and efficient
Panoptic Segmentation in recent times. However, when it comes to adverse
conditions like dark scenes with poor illumination or nighttime images,
existing methods perform poorly in comparison to daytime images. One of the
main factors for poor results is the lack of sufficient and accurately
annotated nighttime images for urban driving scenes. In this work, we propose
two new methods, first to improve the performance, and second to improve the
robustness of panoptic segmentation in nighttime or poor illumination urban
driving scenes using a domain translation approach. The proposed approach makes
use of CycleGAN (Zhu et al., 2017) to translate daytime images with existing
panoptic annotations into nighttime images, which are then utilized to retrain
a Panoptic segmentation model to improve performance and robustness under poor
illumination and nighttime conditions. In our experiments, Approach-1
demonstrates a significant improvement in the Panoptic segmentation performance
on the converted Cityscapes dataset with more than +10% PQ, +12% RQ, +2% SQ,
+14% mIoU and +10% AP50 absolute gain. Approach-2 demonstrates improved
robustness to varied nighttime driving environments. Both the approaches are
supported via comprehensive quantitative and qualitative analysis.