DiffusionSat: A Generative Foundation Model for Satellite Imagery
Samar Khanna Patrick Liu Linqi Zhou Chenlin Meng Robin Rombach Marshall Burke David Lobell Stefano Ermon
Diffusion models have achieved state-of-the-art results on many modalities
including images, speech, and video. However, existing models are not tailored
to support remote sensing data, which is widely used in important applications
including environmental monitoring and crop-yield prediction. Satellite images
are significantly different from natural images -- they can be multi-spectral,
irregularly sampled across time -- and existing diffusion models trained on
images from the Web do not support them. Furthermore, remote sensing data is
inherently spatio-temporal, requiring conditional generation tasks not
supported by traditional methods based on captions or images. In this paper, we
present DiffusionSat, to date the largest generative foundation model trained
on a collection of publicly available large, high-resolution remote sensing
datasets. As text-based captions are sparsely available for satellite images,
we incorporate the associated metadata such as geolocation as conditioning
information. Our method produces realistic samples and can be used to solve
multiple generative tasks including temporal generation, superresolution given
multi-spectral inputs and in-painting. Our method outperforms previous
state-of-the-art methods for satellite image generation and is the first
large-scale $\textit{generative}$ foundation model for satellite imagery.