AnyDoor: Zero-shot Object-level Image Customization
Xi Chen Lianghua Huang Yu Liu Yujun Shen Deli Zhao Hengshuang Zhao
This work presents AnyDoor, a diffusion-based image generator with the power
to teleport target objects to new scenes at user-specified locations in a
harmonious way. Instead of tuning parameters for each object, our model is
trained only once and effortlessly generalizes to diverse object-scene
combinations at the inference stage. Such a challenging zero-shot setting
requires an adequate characterization of a certain object. To this end, we
complement the commonly used identity feature with detail features, which are
carefully designed to maintain texture details yet allow versatile local
variations (e.g., lighting, orientation, posture, etc.), supporting the object
in favorably blending with different surroundings. We further propose to borrow
knowledge from video datasets, where we can observe various forms (i.e., along
the time axis) of a single object, leading to stronger model generalizability
and robustness. Extensive experiments demonstrate the superiority of our
approach over existing alternatives as well as its great potential in
real-world applications, such as virtual try-on and object moving. Project page
is https://damo-vilab.github.io/AnyDoor-Page/.