AnyLens: A Generative Diffusion Model with Any Rendering Lens
Andrey Voynov Amir Hertz Moab Arar Shlomi Fruchter Daniel Cohen-Or
State-of-the-art diffusion models can generate highly realistic images based
on various conditioning like text, segmentation, and depth. However, an
essential aspect often overlooked is the specific camera geometry used during
image capture. The influence of different optical systems on the final scene
appearance is frequently overlooked. This study introduces a framework that
intimately integrates a text-to-image diffusion model with the particular lens
geometry used in image rendering. Our method is based on a per-pixel coordinate
conditioning method, enabling the control over the rendering geometry. Notably,
we demonstrate the manipulation of curvature properties, achieving diverse
visual effects, such as fish-eye, panoramic views, and spherical texturing
using a single diffusion model.