Spatial-Semantic Collaborative Cropping for User Generated Content
Yukun Su Yiwen Cao Jingliang Deng Fengyun Rao Qingyao Wu
A large amount of User Generated Content (UGC) is uploaded to the Internet
daily and displayed to people world-widely through the client side (e.g.,
mobile and PC). This requires the cropping algorithms to produce the aesthetic
thumbnail within a specific aspect ratio on different devices. However,
existing image cropping works mainly focus on landmark or landscape images,
which fail to model the relations among the multi-objects with the complex
background in UGC. Besides, previous methods merely consider the aesthetics of
the cropped images while ignoring the content integrity, which is crucial for
UGC cropping. In this paper, we propose a Spatial-Semantic Collaborative
cropping network (S2CNet) for arbitrary user generated content accompanied by a
new cropping benchmark. Specifically, we first mine the visual genes of the
potential objects. Then, the suggested adaptive attention graph recasts this
task as a procedure of information association over visual nodes. The
underlying spatial and semantic relations are ultimately centralized to the
crop candidate through differentiable message passing, which helps our network
efficiently to preserve both the aesthetics and the content integrity.
Extensive experiments on the proposed UGCrop5K and other public datasets
demonstrate the superiority of our approach over state-of-the-art counterparts.
Our project is available at https://github.com/suyukun666/S2CNet.