Iray Programmer's Manual

Synchronous and streaming video modes

Iray Bridge supports three video modes that have different characteristics and use cases. The modes are controlled by setting the render context option video_mode to either "synchronous", "streaming", or "batch".

Synchronous mode

The synchronous mode is the simplest to understand. It works exactly like local renderers behave, except that it does only support one result canvas at a time. Render is called with a transaction and a rendered frame is returned that reflects the state of that transaction. Iray Bridge achieves this by sending any incremental changes to the server, telling the server to render and encode a frame containing those changes and send it back to the client. The client waits for the frame, decodes it, and copies it into the Canvas passed in the render call. This mode guarantees that the rendered frame matches the state of the transaction passed in the render call, but this guarantee comes at the expense of frame rate. In the synchronous mode, frame rate is determined not only by actual render time but also the server ping time, transmission time of the data over the network, and encoding and decoding.

Synchronous mode is very sensitive to network latency. For instance, if ping time to the server is 100 ms, then it will take a minimum of 200 ms to send incremental changes to the server and get an image back. This gives a maximum possible framerate of 5 fps, and this is before taking any other cost into account. (The streaming video mode is designed to overcome this limitation.)

Note: The framerate of the video stream does not affect convergence time of the rendering. The server will keep rendering full speed in the background, also when frames are encoded and in transit.

The synchronous video mode must be used when the application requires the rendered frame to match the state of the transaction used in the render call. This is the case for instance, when blending the result with other renderers. The synchronous mode is recommended when doing final renderings, together with one of the lossless image formats such as PNG or EXR.

Batch mode

This mode is meant for rendering final images in an offline fashion which does not require interactivity at all. This mode is in many aspects similar to the Synchronous mode.

Rendering is synchronous. In contrast to the Synchronous mode the Batch mode supports the rendering of multiple buffers at the same time. Because of the intended use case and for simplicity all images will automatically be transmitted in a lossless format; it is not possible to specify image formats for the resulting images. This means that the mode is in all cases the most accurate but also the transmission of the images is the slowest. For rendering with the Iray Photoreal render mode, a long "batch_update_interval" should be chosen, and the "batch" scheduler mode should be chosen.

Streaming mode

In contrast to synchronous mode, the streaming video mode allows a certain amount of lag to be able to achieve a higher frame rate and better interactivity. This means that a frame returned by a render call in this mode will not always contain the changes of the passed-in transaction. Instead, the render call streams the incremental changes of the transaction to the server and then returns the latest video frame that has already arrived from the server. The server-side render loop picks up the changes streamed from the client between frames; rendered frames are encoded and sent to the client when ready. This process makes the streaming video mode much less sensitive to network latency and allows it to achieve considerably higher frame rates than synchronous mode.

Since incremental changes are streamed to the server when calling render it is important to call render very often so that recent changes are available to be picked up by the server-side render loop when rendering the next frame. Te render call can be very quick in streaming mode because it will generaly not wait for any actual rendering to be done, it will just copy the latest available frame into the canvas.

Under optimal conditions, the canvas returned by a render call lags behind one frame. Depending on network conditions it can be more, for example when the available bandwidth is lower than the bandwidth is specified for the video stream. The render context option video_maxlag controls the maximum number of frames the server allows the client to lag behind before pausing the video stream to let the client catch up. Setting this to a low value will keep lag at a minimum, but can have a negative impact on framerate since the server might have to pause the video more often. Setting this to a higher value can increase frame rate, but can result in the client lagging more. This setting defaults to 3 as a reasonable compromise, but might have to be set higher if server latency is very high. Note that only the video stream is paused, not the server side render loop, so convergence time is not affected.

The server will send frames as soon as they are rendered to minimize lag, but it will cap the frame rate to the one set in the video_framerate option. The default is 15. If the server-side render loop framerate is higher, it will still only send out video frames at 15 fps. If the frame rate is lower, the server will send out frames as soon as they are ready, allowing the video framerate to drop below the set value. If H.264 is the video format then the server-side H.264 encoder will be configured to use the configured frame rate and bit rate. So if video_bitrate is set to 5-Mbit, then setting the framerate to 10 will give significantly better video quality than setting it to 60, but the frame rate will be capped at 10.

Note: The frame rate of the video stream does not affect the convergence time of the rendered image because the server-side render loop renders as fast as it can in parallel to video frames being encoded and sent on the video stream.

The streaming mode is best used in combination with the H.264 video format to achieve high frame rate and low-latency video streams that make the best use of the available bandwidth. It is perfectly possible to use streaming mode with other formats, such as PNG and EXR, but this will have a negative impact on frame rate because these formats are much more expensive to encode and decode and also take up a lot more bandwidth. Using the streaming mode for final renderings is possible but not recommended; because of the lag, the last frame returned will generally contain a slightly lower number of iterations than reported.