Zooming Out on Zooming In: Advancing Super-Resolution for Remote Sensing
Piper Wolters Favyen Bastani Aniruddha Kembhavi
Super-Resolution for remote sensing has the potential for huge impact on
planet monitoring by producing accurate and realistic high resolution imagery
on a frequent basis and a global scale. Despite a lot of attention, several
inconsistencies and challenges have prevented it from being deployed in
practice. These include the lack of effective metrics, fragmented and
relatively small-scale datasets for training, insufficient comparisons across a
suite of methods, and unclear evidence for the use of super-resolution outputs
for machine consumption. This work presents a new metric for super-resolution,
CLIPScore, that corresponds far better with human judgments than previous
metrics on an extensive study. We use CLIPScore to evaluate four standard
methods on a new large-scale dataset, S2-NAIP, and three existing benchmark
datasets, and find that generative adversarial networks easily outperform more
traditional L2 loss-based models and are more semantically accurate than modern
diffusion models. We also find that using CLIPScore as an auxiliary loss can
speed up the training of GANs by 18x and lead to improved outputs, resulting in
an effective model in diverse geographies across the world which we will
release publicly. The dataset, pre-trained model weights, and code are
available at https://github.com/allenai/satlas-super-resolution/.