HTC-DC Net: Monocular Height Estimation from Single Remote Sensing
  Images
Sining Chen Yilei Shi Zhitong Xiong Xiao Xiang Zhu
3D geo-information is of great significance for understanding the living
environment; however, 3D perception from remote sensing data, especially on a
large scale, is restricted. To tackle this problem, we propose a method for
monocular height estimation from optical imagery, which is currently one of the
richest sources of remote sensing data. As an ill-posed problem, monocular
height estimation requires well-designed networks for enhanced representations
to improve performance. Moreover, the distribution of height values is
long-tailed with the low-height pixels, e.g., the background, as the head, and
thus trained networks are usually biased and tend to underestimate building
heights. To solve the problems, instead of formalizing the problem as a
regression task, we propose HTC-DC Net following the classification-regression
paradigm, with the head-tail cut (HTC) and the distribution-based constraints
(DCs) as the main contributions. HTC-DC Net is composed of the backbone network
as the feature extractor, the HTC-AdaBins module, and the hybrid regression
process. The HTC-AdaBins module serves as the classification phase to determine
bins adaptive to each input image. It is equipped with a vision transformer
encoder to incorporate local context with holistic information and involves an
HTC to address the long-tailed problem in monocular height estimation for
balancing the performances of foreground and background pixels. The hybrid
regression process does the regression via the smoothing of bins from the
classification phase, which is trained via DCs. The proposed network is tested
on three datasets of different resolutions, namely ISPRS Vaihingen (0.09 m),
DFC19 (1.3 m) and GBH (3 m). Experimental results show the superiority of the
proposed network over existing methods by large margins. Extensive ablation
studies demonstrate the effectiveness of each design component.