LoMOE: Localized Multi-Object Editing via Multi-Diffusion
Goirik Chakrabarty Aditya Chandrasekar Ramya Hebbalaguppe Prathosh AP
Recent developments in the field of diffusion models have demonstrated an
exceptional capacity to generate high-quality prompt-conditioned image edits.
Nevertheless, previous approaches have primarily relied on textual prompts for
image editing, which tend to be less effective when making precise edits to
specific objects or fine-grained regions within a scene containing
single/multiple objects. We introduce a novel framework for zero-shot localized
multi-object editing through a multi-diffusion process to overcome this
challenge. This framework empowers users to perform various operations on
objects within an image, such as adding, replacing, or editing $\textbf{many}$
objects in a complex scene $\textbf{in one pass}$. Our approach leverages
foreground masks and corresponding simple text prompts that exert localized
influences on the target regions resulting in high-fidelity image editing. A
combination of cross-attention and background preservation losses within the
latent space ensures that the characteristics of the object being edited are
preserved while simultaneously achieving a high-quality, seamless
reconstruction of the background with fewer artifacts compared to the current
methods. We also curate and release a dataset dedicated to multi-object
editing, named $\texttt{LoMOE}$-Bench. Our experiments against existing
state-of-the-art methods demonstrate the improved effectiveness of our approach
in terms of both image editing quality and inference speed.