Matte objects and matte lights

Introduction

Matte objects and matte lights are advanced techniques to aid including synthetic objects under matching lighting (captured for example via lightprobes) onto a real world photograph (a virtual backplate or environment map). In a classic workflow, one needs to assign special custom shaders to objects to receive a separate shadow pass along with additional global illumination information that must then be combined with the original rendering pass in external compositing steps.

Iray Photoreal simplifies this classic workflow by avoiding the separate postprocessing steps and allowing for a fully progressive and physically plausible rendering of the complete scene at once. In addition, matte objects do not need to rely on secondary buffers and setting up the additional shaders that must usually be tweaked on a per scene basis, as all of the necessary logic is directly incorporated into the actual core rendering architecture.

To guide the interaction of backplate and rendered objects though, additional matte objects have to be provided that should (roughly) match the photographed real world objects using the same camera settings. In addition, these objects should be assigned (roughly) matching materials to further perfect the illusion of the rendered objects being part of the environment and especially interacting closely with the backplate.

Attribute

The following attribute on the mi::IAttribute_set class controls matte objects:

bool matte = false

If set to true, this attribute flags objects and lights as matte objects or matte lights, respectively.

Example

A simple example would be to add a synthetic cube onto a backplate of a photographed wooden table.

• The scene must be set up to include both the cube and the top of the table (which can be a very rough approximation, so a simple rectangle the size of the table can already be sufficient).
• The virtual camera must be set up to match the original real world camera position and lens settings (to have the objects in scene and photograph align).
• An environment map with similar lighting characteristics (best would be a matching lightprobe of the same real world scene) must be provided.
• Approximately match the material of the original table, including textures (or even a bump or normal map) and enabling the matte flag of the table object.

Instead of rendering the table object directly into the output buffer like any other standard object, Iray Photoreal now triggers a specialized handling of the matte object that directly combines backplate information, environment lighting, interactions of both matte and synthetic objects (global illumination) and the artificial shadows cast onto the matte objects, in a single progressive rendering step.

Additional notes and limitations

An environment map or a backplate (or both) should be specified to get the full benefit of matte objects. These define the appearance of matte objects when they interact (through reflections and refractions) with synthetic objects. The backplate is projected onto the matte object from the camera. If this projection is impossible, the environment map is used instead. If a hemispherical environment dome of sufficient resolution and quality is available, this can also be used alone. But these are difficult to generate, so in most cases a 2D photograph will be used as the backplate, which can be seen as a high-quality complement to the environment map.

It is possible to use the matte objects feature with neither a backplate nor an environment map. In such a case, matte lights should be used to approximate the scene lighting. But many interesting effects will then be missing from the rendered image, such as reflections of matte objects onto synthetic objects, because the backplate/environment defines how matte objects appear in those reflections.