APNet: Urban-level Scene Segmentation of Aerial Images and Point Clouds
Weijie Wei Martin R. Oswald Fatemeh Karimi Nejadasl Theo Gevers
In this paper, we focus on semantic segmentation method for point clouds of
urban scenes. Our fundamental concept revolves around the collaborative
utilization of diverse scene representations to benefit from different context
information and network architectures. To this end, the proposed network
architecture, called APNet, is split into two branches: a point cloud branch
and an aerial image branch which input is generated from a point cloud. To
leverage the different properties of each branch, we employ a geometry-aware
fusion module that is learned to combine the results of each branch. Additional
separate losses for each branch avoid that one branch dominates the results,
ensure the best performance for each branch individually and explicitly define
the input domain of the fusion network assuring it only performs data fusion.
Our experiments demonstrate that the fusion output consistently outperforms the
individual network branches and that APNet achieves state-of-the-art
performance of 65.2 mIoU on the SensatUrban dataset. Upon acceptance, the
source code will be made accessible.