AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene
  Graph Generation
Yansheng Li Kun Li Yongjun Zhang Linlin Wang Dingwen Zhang
Scene graph generation (SGG) aims to understand the visual objects and their
semantic relationships from one given image. Until now, lots of SGG datasets
with the eyelevel view are released but the SGG dataset with the overhead view
is scarcely studied. By contrast to the object occlusion problem in the
eyelevel view, which impedes the SGG, the overhead view provides a new
perspective that helps to promote the SGG by providing a clear perception of
the spatial relationships of objects in the ground scene. To fill in the gap of
the overhead view dataset, this paper constructs and releases an aerial image
urban scene graph generation (AUG) dataset. Images from the AUG dataset are
captured with the low-attitude overhead view. In the AUG dataset, 25,594
objects, 16,970 relationships, and 27,175 attributes are manually annotated. To
avoid the local context being overwhelmed in the complex aerial urban scene,
this paper proposes one new locality-preserving graph convolutional network
(LPG). Different from the traditional graph convolutional network, which has
the natural advantage of capturing the global context for SGG, the
convolutional layer in the LPG integrates the non-destructive initial features
of the objects with dynamically updated neighborhood information to preserve
the local context under the premise of mining the global context. To address
the problem that there exists an extra-large number of potential object
relationship pairs but only a small part of them is meaningful in AUG, we
propose the adaptive bounding box scaling factor for potential relationship
detection (ABS-PRD) to intelligently prune the meaningless relationship pairs.
Extensive experiments on the AUG dataset show that our LPG can significantly
outperform the state-of-the-art methods and the effectiveness of the proposed
locality-preserving strategy.