IterInv: Iterative Inversion for Pixel-Level T2I Models
Chuanming Tang Kai Wang Joost van de Weijer
Large-scale text-to-image diffusion models have been a ground-breaking
development in generating convincing images following an input text prompt. The
goal of image editing research is to give users control over the generated
images by modifying the text prompt. Current image editing techniques are
relying on DDIM inversion as a common practice based on the Latent Diffusion
Models (LDM). However, the large pretrained T2I models working on the latent
space as LDM suffer from losing details due to the first compression stage with
an autoencoder mechanism. Instead, another mainstream T2I pipeline working on
the pixel level, such as Imagen and DeepFloyd-IF, avoids this problem. They are
commonly composed of several stages, normally with a text-to-image stage
followed by several super-resolution stages. In this case, the DDIM inversion
is unable to find the initial noise to generate the original image given that
the super-resolution diffusion models are not compatible with the DDIM
technique. According to our experimental findings, iteratively concatenating
the noisy image as the condition is the root of this problem. Based on this
observation, we develop an iterative inversion (IterInv) technique for this
stream of T2I models and verify IterInv with the open-source DeepFloyd-IF
model. By combining our method IterInv with a popular image editing method, we
prove the application prospects of IterInv. The code will be released at
\url{https://github.com/Tchuanm/IterInv.git}.