HeightFormer: A Multilevel Interaction and Image-adaptive
  Classification-regression Network for Monocular Height Estimation with Aerial
  Images
Zhan Chen Yidan Zhang Xiyu Qi Yongqiang Mao Xin Zhou Lulu Niu Hui Wu Lei Wang Yunping Ge
Height estimation has long been a pivotal topic within measurement and remote
sensing disciplines, proving critical for endeavours such as 3D urban
modelling, MR and autonomous driving. Traditional methods utilise stereo
matching or multisensor fusion, both well-established techniques that typically
necessitate multiple images from varying perspectives and adjunct sensors like
SAR, leading to substantial deployment costs. Single image height estimation
has emerged as an attractive alternative, boasting a larger data source variety
and simpler deployment. However, current methods suffer from limitations such
as fixed receptive fields, a lack of global information interaction, leading to
noticeable instance-level height deviations. The inherent complexity of height
prediction can result in a blurry estimation of object edge depth when using
mainstream regression methods based on fixed height division. This paper
presents a comprehensive solution for monocular height estimation in remote
sensing, termed HeightFormer, combining multilevel interactions and
image-adaptive classification-regression. It features the Multilevel
Interaction Backbone (MIB) and Image-adaptive Classification-regression Height
Generator (ICG). MIB supplements the fixed sample grid in CNN of the
conventional backbone network with tokens of different interaction ranges. It
is complemented by a pixel-, patch-, and feature map-level hierarchical
interaction mechanism, designed to relay spatial geometry information across
different scales and introducing a global receptive field to enhance the
quality of instance-level height estimation. The ICG dynamically generates
height partition for each image and reframes the traditional regression task,
using a refinement from coarse to fine classification-regression that
significantly mitigates the innate ill-posedness issue and drastically improves
edge sharpness.