arxiv:2505.09608

LightLab: Controlling Light Sources in Images with Diffusion Models

Published on May 14

· Submitted by

NadMag on May 15

Upvote

Authors:

Nadav Magar ,

Abstract

We present a simple, yet effective diffusion-based method for fine-grained, parametric control over light sources in an image. Existing relighting methods either rely on multiple input views to perform inverse rendering at inference time, or fail to provide explicit control over light changes. Our method fine-tunes a diffusion model on a small set of real raw photograph pairs, supplemented by synthetically rendered images at scale, to elicit its photorealistic prior for relighting. We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination. Using this data and an appropriate fine-tuning scheme, we train a model for precise illumination changes with explicit control over light intensity and color. Lastly, we show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.

View arXiv page View PDF Project page Add to collection

Community

NadMag

Paper author Paper submitter about 14 hours ago

We recently published a research paper titled: "LightLab: Controlling Light Sources in Images with Diffusion Models" (accepted to SIGGRAPH 25).
In the paper we demonstrate how to achieve control over visible (and ambient) light sources from a single image.
The premise of the paper is that you can generate physically-accurate training data using classic computational photography, from paired images depicting a change in a visible light source.
We also study how supplementing this small set with synthetic renders affects the trained model's results.
Personally the quality of the results was surprising due to the simplicity of the method, and the relatively low-diversity dataset, which is used in a smart manner.

Project page: https://nadmag.github.io/LightLab/