Open-World Semantic Segmentation Including Class Similarity
Matteo Sodano Federico Magistri Lucas Nunes Jens Behley Cyrill Stachniss
Interpreting camera data is key for autonomously acting systems, such as
autonomous vehicles. Vision systems that operate in real-world environments
must be able to understand their surroundings and need the ability to deal with
novel situations. This paper tackles open-world semantic segmentation, i.e.,
the variant of interpreting image data in which objects occur that have not
been seen during training. We propose a novel approach that performs accurate
closed-world semantic segmentation and, at the same time, can identify new
categories without requiring any additional training data. Our approach
additionally provides a similarity measure for every newly discovered class in
an image to a known category, which can be useful information in downstream
tasks such as planning or mapping. Through extensive experiments, we show that
our model achieves state-of-the-art results on classes known from training data
as well as for anomaly segmentation and can distinguish between different
unknown classes.