Detecting Cloud Presence in Satellite Images Using the RGB-based CLIP
  Vision-Language Model
Mikolaj Czerkawski Robert Atkinson Christos Tachtatzis
This work explores capabilities of the pre-trained CLIP vision-language model
to identify satellite images affected by clouds. Several approaches to using
the model to perform cloud presence detection are proposed and evaluated,
including a purely zero-shot operation with text prompts and several
fine-tuning approaches. Furthermore, the transferability of the methods across
different datasets and sensor types (Sentinel-2 and Landsat-8) is tested. The
results that CLIP can achieve non-trivial performance on the cloud presence
detection task with apparent capability to generalise across sensing modalities
and sensing bands. It is also found that a low-cost fine-tuning stage leads to
a strong increase in true negative rate. The results demonstrate that the
representations learned by the CLIP model can be useful for satellite image
processing tasks involving clouds.