Continuous-Multiple Image Outpainting in One-Step via Positional Query
  and A Diffusion-based Approach
Shaofeng Zhang Jinfa Huang Qiang Zhou Zhibin Wang Fan Wang Jiebo Luo Junchi Yan
Image outpainting aims to generate the content of an input sub-image beyond
its original boundaries. It is an important task in content generation yet
remains an open problem for generative models. This paper pushes the technical
frontier of image outpainting in two directions that have not been resolved in
literature: 1) outpainting with arbitrary and continuous multiples (without
restriction), and 2) outpainting in a single step (even for large expansion
multiples). Moreover, we develop a method that does not depend on a pre-trained
backbone network, which is in contrast commonly required by the previous SOTA
outpainting methods. The arbitrary multiple outpainting is achieved by
utilizing randomly cropped views from the same image during training to capture
arbitrary relative positional information. Specifically, by feeding one view
and positional embeddings as queries, we can reconstruct another view. At
inference, we generate images with arbitrary expansion multiples by inputting
an anchor image and its corresponding positional embeddings. The one-step
outpainting ability here is particularly noteworthy in contrast to previous
methods that need to be performed for $N$ times to obtain a final multiple
which is $N$ times of its basic and fixed multiple. We evaluate the proposed
approach (called PQDiff as we adopt a diffusion-based generator as our
embodiment, under our proposed \textbf{P}ositional \textbf{Q}uery scheme) on
public benchmarks, demonstrating its superior performance over state-of-the-art
approaches. Specifically, PQDiff achieves state-of-the-art FID scores on the
Scenery (\textbf{21.512}), Building Facades (\textbf{25.310}), and WikiArts
(\textbf{36.212}) datasets. Furthermore, under the 2.25x, 5x and 11.7x
outpainting settings, PQDiff only takes \textbf{40.6\%}, \textbf{20.3\%} and
\textbf{10.2\%} of the time of the benchmark state-of-the-art (SOTA) method.