Multi-task deep learning for large-scale building detail extraction from
  high-resolution satellite imagery
Zhen Qian Min Chen Zhuo Sun Fan Zhang Qingsong Xu Jinzhao Guo Zhiwei Xie Zhixin Zhang
Understanding urban dynamics and promoting sustainable development requires
comprehensive insights about buildings. While geospatial artificial
intelligence has advanced the extraction of such details from Earth
observational data, existing methods often suffer from computational
inefficiencies and inconsistencies when compiling unified building-related
datasets for practical applications. To bridge this gap, we introduce the
Multi-task Building Refiner (MT-BR), an adaptable neural network tailored for
simultaneous extraction of spatial and attributional building details from
high-resolution satellite imagery, exemplified by building rooftops, urban
functional types, and roof architectural types. Notably, MT-BR can be
fine-tuned to incorporate additional building details, extending its
applicability. For large-scale applications, we devise a novel spatial sampling
scheme that strategically selects limited but representative image samples.
This process optimizes both the spatial distribution of samples and the urban
environmental characteristics they contain, thus enhancing extraction
effectiveness while curtailing data preparation expenditures. We further
enhance MT-BR's predictive performance and generalization capabilities through
the integration of advanced augmentation techniques. Our quantitative results
highlight the efficacy of the proposed methods. Specifically, networks trained
with datasets curated via our sampling method demonstrate improved predictive
accuracy relative to those using alternative sampling approaches, with no
alterations to network architecture. Moreover, MT-BR consistently outperforms
other state-of-the-art methods in extracting building details across various
metrics. The real-world practicality is also demonstrated in an application
across Shanghai, generating a unified dataset that encompasses both the spatial
and attributional details of buildings.