Physically Controllable Relighting of Photographs

This SIGGRAPH 2025 work presents a way to relight an ordinary photograph with the explicit, 3D-style control found in computer graphics software. Rather than applying a global color or brightness edit, users can add and configure different light sources - including spotlights, point lights, and environmental illumination - and see how they affect the pictured scene. The method first estimates scene geometry and image components such as surface color and shading from a single photograph, then builds a colored mesh that can be illuminated in 3D. A path-tracing renderer computes an approximate image under the requested lighting.

Because that physically based rendering does not fully capture the appearance of a real photograph, a feed-forward neural renderer refines it into a more photorealistic result. The researchers address the challenge of training this network without paired examples of real scenes under known lighting: differentiable rendering lets them infer illumination in real-world photographs and train from raw image collections through self-supervision. The approach thus combines explicit, physically meaningful light editing with neural correction intended to generalize to in-the-wild images. Its central contribution is a pipeline that makes lighting changes controllable in a way conventional image edits are not, while using learned rendering to compensate for imperfect scene estimates and approximate physical renders.