Quantifying the robustness of deep multispectral segmentation models
  against natural perturbations and data poisoning
Elise Bishoff Charles Godfrey Myles McKay Eleanor Byler
In overhead image segmentation tasks, including additional spectral bands
beyond the traditional RGB channels can improve model performance. However, it
is still unclear how incorporating this additional data impacts model
robustness to adversarial attacks and natural perturbations. For adversarial
robustness, the additional information could improve the model's ability to
distinguish malicious inputs, or simply provide new attack avenues and
vulnerabilities. For natural perturbations, the additional information could
better inform model decisions and weaken perturbation effects or have no
significant influence at all. In this work, we seek to characterize the
performance and robustness of a multispectral (RGB and near infrared) image
segmentation model subjected to adversarial attacks and natural perturbations.
While existing adversarial and natural robustness research has focused
primarily on digital perturbations, we prioritize on creating realistic
perturbations designed with physical world conditions in mind. For adversarial
robustness, we focus on data poisoning attacks whereas for natural robustness,
we focus on extending ImageNet-C common corruptions for fog and snow that
coherently and self-consistently perturbs the input data. Overall, we find both
RGB and multispectral models are vulnerable to data poisoning attacks
regardless of input or fusion architectures and that while physically
realizable natural perturbations still degrade model performance, the impact
differs based on fusion architecture and input data.