Modeling Weather Uncertainty for Multi-weather Co-Presence Estimation
Qi Bi Shaodi You Theo Gevers
Images from outdoor scenes may be taken under various weather conditions. It
is well studied that weather impacts the performance of computer vision
algorithms and needs to be handled properly. However, existing algorithms model
weather condition as a discrete status and estimate it using multi-label
classification. The fact is that, physically, specifically in meteorology,
weather are modeled as a continuous and transitional status. Instead of
directly implementing hard classification as existing multi-weather
classification methods do, we consider the physical formulation of
multi-weather conditions and model the impact of physical-related parameter on
learning from the image appearance. In this paper, we start with solid revisit
of the physics definition of weather and how it can be described as a
continuous machine learning and computer vision task. Namely, we propose to
model the weather uncertainty, where the level of probability and co-existence
of multiple weather conditions are both considered. A Gaussian mixture model is
used to encapsulate the weather uncertainty and a uncertainty-aware
multi-weather learning scheme is proposed based on prior-posterior learning. A
novel multi-weather co-presence estimation transformer (MeFormer) is proposed.
In addition, a new multi-weather co-presence estimation (MePe) dataset, along
with 14 fine-grained weather categories and 16,078 samples, is proposed to
benchmark both conventional multi-label weather classification task and
multi-weather co-presence estimation task. Large scale experiments show that
the proposed method achieves state-of-the-art performance and substantial
generalization capabilities on both the conventional multi-label weather
classification task and the proposed multi-weather co-presence estimation task.
Besides, modeling weather uncertainty also benefits adverse-weather semantic
segmentation.