Image based lighting (IBL) is a technique for capturing real world lighting conditions and using this information to create photo-realistic renderings of virtual objects such that they appear to be placed into the real scene. The real-world lighting is usually captured as a single 360 degree panoramic high dynamic range (HDR) image. Our research makes it possible to capture and use time varying lighting and spatial variations in the scene illumination.

An HDR image covers the full dynamic range of the lighting conditions in a scene. By capturing the image as a panorama, it describes the lighting incident from any direction onto the capture position as exemplified by the images with the toy cars. Image based lighting is one of the key applications of HDR imaging. Here, we display how IBL can be extended through the use of HDRv sequences. The benefit of using HDRv sequences instead of single HDR images (as in ordinary IBL) is that HDRv makes it possible to capture time varying lighting conditions or spatial variations in the scene illumination.

A collage of several images where one is a spherical panorama of an outdoor environment and the others picture a black and a whit toy car in different lightingThe image displays an IBL example with the panoramic HDR image in the top left and three renderings where the captured lighting information is used to illuminate the virtual toy cars.

 

Traditional IBL compared to HDRv sequencing

Comparison between different types of image based lightning The image above displays a comparison between traditional IBL using a single light probe (left) and using a lighting environment including 3D geometry that was built from a large number of HDR light probe images captured from an HDRv sequence (right). It is evident that the traditional IBL method cannot capture the spatial variations in the scene illumination. This can be seen from the shadow cast by the window and reflection of the light sources in the photograph on the wall behind the sofa. The scene was modeled by Sören Larsson, RayLab.

Scene and illumination capture for IBL

The IBL scene capture is based on a 4 Mpixel global shutter HDRv camera with a dynamic range of more than 10.000.000 : 1 at 30 fps. The image displays the HDRv camera in a real-time light probe (RTLP) setup. The camera is capturing the environment through the reflection in a mirror sphere. This yields a close to 360 degree panorama at each capture position.Video camera and light probe

A scene is usually captured using a combination of panoramic light probe sequences, and sequences with a smaller field of view to maximize the resolution at regions of special interest in the scene. The panoramic sequences ensure full angular coverage at each position and guarantee that the information required for IBL is captured. The system yields high resolution HDR environment maps that can be directly used for IBL rendering or scene reconstruction.
 
The image below displays an older RTLP-system developed in a collaboration with the camera manufacturer SICK-IVP. The first version, implemented in 2003, was capable of capturing images with dynamic range of 10.000.000 : 1 and a resolution of 960 x 512 pixels at 25 frames per second. 

Video camera and light probe

This imaging system has been used in several projects directed towards capturing, processing and rendering with spatially varying lighting conditions. We call this extension of IBL: Incident Light Fields (ILF).

The image below displays one of the first renderings using captured and reconstructed real-world lighting exhibiting strong spatial variations. The lighting was captured using around 600 HDR light probe images distributed along a tracked 1D path in space. The tracking information makes it possible to geometrically relate the light probes and reconstruct the sharp shadows created by a blocker in front of the projector light source.

A figure and several red and silver colored spheres in different lighting

HDRv light probe sequence used for image based lighting

The video displays an HDRv sequence of a mirror sphere (left) capturing a panoramic image of the environment. Each HDR video frame accurately captures the scene lighting in the scene and is used as source of illumination in the corresponding rendering to the right.

Cover of publication ''
Jonas Unger, Joel Kronander, Per Larsson, Stefan Gustavson, Joakim Löw, Anders Ynnerman (2013)

Computers & graphics , Vol.37 , s.923-934 Continue to DOI

Cover of publication ''
Jonas Unger, Joel Kronander, Per Larsson, Stefan Gustavson, Anders Ynner (2013)

Proceedings of the 21st European Signal Processing Conference (EUSIPCO), 2013: Special Session on HDR-video , s.1-5

Cover of publication ''
Jonas Unger (2009)
Cover of publication ''
Jonas Unger, Stefan Gustavson, Larsson Per, Anders Ynnerman (2008)

Computer graphics forum (Print) , Vol.27 , s.1293-1301 Continue to DOI