Beyond Image Borders: Learning Feature Extrapolation for Unbounded Image
  Composition
Xiaoyu Liu Ming Liu Junyi Li Shuai Liu Xiaotao Wang Lei Lei Wangmeng Zuo
For improving image composition and aesthetic quality, most existing methods
modulate the captured images by striking out redundant content near the image
borders. However, such image cropping methods are limited in the range of image
views. Some methods have been suggested to extrapolate the images and predict
cropping boxes from the extrapolated image. Nonetheless, the synthesized
extrapolated regions may be included in the cropped image, making the image
composition result not real and potentially with degraded image quality. In
this paper, we circumvent this issue by presenting a joint framework for both
unbounded recommendation of camera view and image composition (i.e., UNIC). In
this way, the cropped image is a sub-image of the image acquired by the
predicted camera view, and thus can be guaranteed to be real and consistent in
image quality. Specifically, our framework takes the current camera preview
frame as input and provides a recommendation for view adjustment, which
contains operations unlimited by the image borders, such as zooming in or out
and camera movement. To improve the prediction accuracy of view adjustment
prediction, we further extend the field of view by feature extrapolation. After
one or several times of view adjustments, our method converges and results in
both a camera view and a bounding box showing the image composition
recommendation. Extensive experiments are conducted on the datasets constructed
upon existing image cropping datasets, showing the effectiveness of our UNIC in
unbounded recommendation of camera view and image composition. The source code,
dataset, and pretrained models is available at
https://github.com/liuxiaoyu1104/UNIC.