Interactive Neural Painting
Elia Peruzzo Willi Menapace Vidit Goel Federica Arrigoni Hao Tang Xingqian Xu Arman Chopikyan Nikita Orlov Yuxiao Hu Humphrey Shi Nicu Sebe Elisa Ricci
In the last few years, Neural Painting (NP) techniques became capable of
producing extremely realistic artworks. This paper advances the state of the
art in this emerging research domain by proposing the first approach for
Interactive NP. Considering a setting where a user looks at a scene and tries
to reproduce it on a painting, our objective is to develop a computational
framework to assist the users creativity by suggesting the next strokes to
paint, that can be possibly used to complete the artwork. To accomplish such a
task, we propose I-Paint, a novel method based on a conditional transformer
Variational AutoEncoder (VAE) architecture with a two-stage decoder. To
evaluate the proposed approach and stimulate research in this area, we also
introduce two novel datasets. Our experiments show that our approach provides
good stroke suggestions and compares favorably to the state of the art.
Additional details, code and examples are available at
https://helia95.github.io/inp-website.