A core concept in RealityServer which many new users have some difficulty understanding is Scopes. The use of scopes is critical in making effective use of RealityServer in a production environment where multiple users or multiple independent operations are happening at once. In this article we will go into more depth on what scopes are and how to use them.
Before diving into scopes you really need to understand some of the terms we will use in relation to how data is stored and used in RealityServer. These terms are utilised throughout the documentation and it is important to have an understanding of what they mean. The main three you need to know are Database, Elements and Attributes.
RealityServer includes a unique, in-memory database which holds all of the information that is loaded and used at any given time. Unlike a traditional database, this resides entirely in server memory and is not persisted to disk. You will not see the database referred to in the documentation often except for the C++ and Iray API since you rarely need to directly manipulate it when using RealityServer normally.
It is difficult to stress just how unique RealityServer’s database system is and much of which relates to its handling of scopes. We are not currently aware of any other similar database for 3D applications which operates in this way and it is one of the reasons that adapting other renderers for deployment as a multi-user web service doesn’t really work. Once you’ve seen the power of scopes below it should become clear why this is needed.
The RealityServer database stores Elements. An element is an instance of a specific type of data which RealityServer can use. You can always find all of the supported element types in the documentation for the create_element command. Some examples of common element types are:
There are additional types which are created and managed by RealityServer as well as some types which Iray supports that you can only access through the C++ API. However as you can see from the above list, these elements are the most common types of things you access in the course of using RealityServer.
All elements in the database must have a name. When you use a command that wants the name of an element (e.g., element_set_attribute) as a parameter, this is the name used to find the element in the database.
All elements in the database can additionally have Attributes associated with them. Attributes are named properties with values that you can store on a given element. There are many standard attributes which are recognised and used by Iray or RealityServer operations, however an attribute is completely generic and you can create your own to store whatever you need.
Attributes are created on an element in the database using the element_set_attribute command and you can see a full list of the supported types in the documentaiton for that command. Basically elements are the things and attributes are the properties of those things.
Chances are you are familiar with the concept of scope from programming, at least for things like variables. In RealityServer a scope defines a named container in which elements within the database live. Only elements and operations within that scope (or a more public scope) can see each other. All elements in the database have a scope (even if that is the global scope).
So what do scopes give us and why do we want to use them? Basically anytime you want to share or restrict the sharing of data between multiple users or operations you will need to use scopes. This might be so different users can both navigate the same model but see different views, or creating shared scopes to cache data used by all users.
Scopes are managed with a few simple commands. To create a new scope you use the create_scope command like this:
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "Fancy_scope"
}, "id": 1}
This creates a new scope under the global scope called Fancy_scope. You can also create child scopes (see the next section for full details), like this:
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"parent_scope" : "Fancy_scope",
"scope_name" : "Child_of_fancy_scope"
}, "id": 1}
You then use a given scope with the use_scope command. It doesn’t make any sense to do this as a single command since what it does is switch the scope to be used for subsequent commands in the same batch. So, typically it is used in a batch, like this:
[
{"jsonrpc": "2.0", "method": "use_scope", "params": {
"scope_name" : "Child_of_fancy_scope"
}, "id": 1},
{"jsonrpc": "2.0", "method": "import_scene", "params": {
"scene_name" : "Fancy_scene",
"filename" : "scenes/meyemII.mi"
}, "id": 2}
]
This is a batch of two commands, the first switches the scope for the batch to Child_of_fancy_scope and then calls the import_scene command in that scope. The result is that all of the elements created by the import_scene command will be placed in the Child_of_fancy_scope scope.
You can also switch scopes multiple times within a batch of commands. Calling use_scope each time can be tedious if you are using the same scope for the lifetime of your application so there are ways to automate this which we will see later.
An important feature of scopes in RealityServer is that they can be nested. When you create a scope with the create_scope command you can specify the optional parent_scope parameter in order to nest the new scope under an existing scope. By default scopes are parented to the global scope. This nesting allows you to create scope structures like this.
This is a very common structure for a RealityServer application but certainly not the only possible one. The following JSON-RPC command sequence would create this structure.
[
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "Application Scope A"
}, "id": 1},
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "User Scope A",
"parent_scope" : "Application Scope A"
}, "id": 2},
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "User Scope B",
"parent_scope" : "Application Scope A"
}, "id": 3},
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "Application Scope B"
}, "id": 4},
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "User Scope C",
"parent_scope" : "Application Scope B"
}, "id": 5},
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "User Scope D",
"parent_scope" : "Application Scope B"
}, "id": 6}
]
Now when you call use_scope with a scope before executing your commands, those commands will have access to all of the elements within that scope and all of its parent scopes. Note that scope names must be globally unique, so you cannot have two scopes with the same name, even if they have different parents.
Any new elements you create will go into the scope you are using. For example, in this case, if you call:
[
{"jsonrpc": "2.0", "method": "use_scope", "params": {
"scope_name" : "User Scope D"
}, "id": 1},
...
]
Then the other commands in the batch will have access to all elements in User Scope D, Application Scope B and the global scope. However they will not see any elements in Application Scope A or User Scope A/B/C.
These nested scopes effectively form a tree. The maximum depth of the scope tree is 255 (including the global scope). However we have never encountered an application that needed this amount. Most applications will only need between 2 – 4 levels.
So far we have just described scopes as a way of isolating elements from each other. However one of the most powerful features of scopes in RealityServer is localization. What this allows you to do is to make a copy of just a given element that exists in a scope your current scope has access to, within your current scope.
Doesn’t sound so interesting at first. However consider this scenario. Let’s say you have a complex scene loaded, for example using 3GB of geometry, images, textures and other elements. Let’s load that into our application scope:
[
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"scope_name" : "Application Scope A"
}, "id": 1},
{"jsonrpc": "2.0", "method": "use_scope", "params": {
"scope_name" : "Application Scope A"
}, "id": 2},
{"jsonrpc": "2.0", "method": "import_scene", "params": {
"scene_name" : "Shared Scene A",
"filename" : "scenes/BigScene.mi"
}, "id": 3}
]
Now you want to have two different users access this scene but you want each user to be able to have a different camera location and change the color of a material on an object in the scene. In a conventional system you would basically run two full copies of the data and consume the memory associated with that. Instead we can do this:
[
{"jsonrpc": "2.0", "method": "create_scope", "params": {
"parent_scope" : "Application Scope A",
"scope_name" : "User Scope A"
}, "id": 1},
{"jsonrpc": "2.0", "method": "use_scope", "params": {
"scope_name" : "User Scope A"
}, "id": 2},
{"jsonrpc": "2.0", "method": "localize_element", "params": {
"element_name" : "BigScene_camera_instance"
}, "id": 3},
{"jsonrpc": "2.0", "method": "localize_element", "params": {
"element_name" : "Material_in_BigScene"
}, "id": 4}
]
In RealityServer you can use localize_element to make a local copy of only the elements you wish to change into your scope. When you make your changes they then will not be visible to other scopes. Here we have made a new child scope and localized our elements to it.
This means instead of duplicating 3GB of data in our example we can just duplicate the hundreds of bytes associated with the camera and material for each user. Then we can make changes in the usual in our user scope and only that user will see the changes:
[
{"jsonrpc": "2.0", "method": "use_scope", "params": {
"scope_name" : "User Scope A"
}, "id": 1},
{"jsonrpc": "2.0", "method": "instance_set_world_to_obj", "params": {
"instance_name" : "BigScene_camera_instance",
"transform" : {
...
}
}, "id": 2},
{"jsonrpc": "2.0", "method": "mdl_set_argument", "params": {
"element_name" : "Material_in_BigScene",
"argument_name" : "diffuse_color",
"value" : { "r": 1.0, "g": 0.0, "b": 0.0}
}, "id": 3}
]
Localization is perhaps the most important aspect to understand and use. If you do not localize elements to a more private scope before changing them, then all users may see the changes.
For most applications, you will want at least two levels of scope. An application scope which holds all data which is not specific to a given user and a user scope which holds any data which a specific user can modify.
If you have models, textures and other data that are always shared and you want to keep them loaded all the time, then import or create them in the application scope.
As soon as you want to change an element for a given user, create and use a user scope and then localize that element by calling the localize_element command on that element.
Typically you will create a new user scope whenever a user navigates to your application. Since they need to be unique we usually named them with a UUID or long random string.
The above strategy is used by the Simple JavaScript Render Loop Rendering Sample that ships with RealityServer if you want to see it in action. Of course you may have different use cases where another structure is appropriate. A common one may be creating view scopes under the user scope if an application has multiple views of the same model.
Finally, note that the rendering commands, render loop and other rendering functionality are all scope aware. So they will render the version of the scene defined by the scope you are in.
You can see that scopes are extremely useful and you will want to be using them in almost every RealityServer application. However it can also become repetitive to send a use_scope command for every set of commands you want to run. If you forget it can also be difficult to debug the issue as you will find your commands can’t access elements you created inside scopes (since the command is running in the global scope). RealityServer has several features to help.
If you are using the JavaScript Client Library in your application (this is used by the examples that ship with RealityServer), then you can set custom StateData which will apply a specific command to each set of requests. You can apply this globally to the RSService object which will use it for all requests, like this:
const service = new com.mi.rs.RSService('127.0.0.1', 8080);
/* ... */
let state_data = new com.mi.rs.StateData(null, null, [
new com.mi.rs.Command("use_scope", {scope_name:'Fancy_scope'})
]);
service.defaultStateData = state_data;
Or you can also define it on a CommandSequence to just apply there. See the JavaScript Client Library documentation for com.mi.rs.StateData for full details.
This plugin comes with the RealityServer Extras plugins set and allows you to control scope through HTTP headers instead of explicitly writing out a use_scope command each time. It is a state handler plugin that automatically uses a specified scope for a request and if the scope does not exist it creates it, so you don’t have to use create_scope either (unless you are making child scopes). You can simply set a HTTP header and if present it will generate and use the scopes. Configure as follows in your realityserver.conf file after installing the plugin:
<url .*> state auto_scope </url> <user auto_scope> header rs-scope </user>
RealityServer will then look for the HTTP header rs-scope and if found will use it to set the current scope for commands or create it if it doesn’t exist. To use this you will naturally need to be using a client that allows you to control the headers of your request (most clients should allow this).
Custom V8 commands on the server-side will execute in the scope they are called in. So if you call one in a batch that has a use_scope call before it, then the V8 command will run in that scope. However you can also call scope commands from within V8 to change the scope during execution. For example:
RS.use_scope({ scope_name: 'Fancy_scope' });
Two other scope commands not discussed yet can also be very useful in V8, namely get_scope which returns the name of the current scope and scope_exists which checks if a scope of a given name already exists. Here is a quick example of how that might be useful in V8:
let original_scope = null;
if (RS.scope_exists({ scope_name: 'Fancy_scope' })) {
original_scope = RS.get_scope();
RS.use_scope({ scope_name: 'Fancy_scope' });
} else {
throw new RS.Error(RS.Error.INVALID_PARAMETERS,
'Fancy_scope does not exist');
}
/* .. */
RS.use_scope({ scope_name: original_scope });
This checks if a required scope exists and if it does uses it but first stores the name of the current scope so it can be restored later on. In general a V8 command should always restore the scope it had when it started to avoid unexpected side-effects for the user.
If you are using RealityServer 5.2 build 2272.287 or later then you have access to the new Scope class which makes this process even simpler in V8. Using this class the above code would become:
const Scope = require('Scope');
/* ... */
let original_scope = Scope.current();
let new_scope = new Scope('Fancy_scope');
if (new_scope.exists) {
new_scope.use();
} else {
throw new RS.Error(RS.Error.INVALID_PARAMETERS,
'Fancy_scope does not exist');
}
/* ... */
original_scope.use();
Please refer to the Server-side V8 documentation for full details. We definitely recommend using the Scope class instead of directly calling the related commands.
There is some information about the RealityServer database which is not possible to obtain through the API. For example a list of all of the elements in the database, or all of the currently defined scopes. This data can however be obtained through the Admin Console.
By default this can be accessed in your browser by navigating to port 8081 on your server (unless you have changed the configuration). It is a very barebones diagnostic tool but does allow you to see all of the currently defined scopes and all of the elements in the database.
This tool can be extremely helpful when diagnosing scoping issues or just finding out what RealityServer currently has in its database.
Just like our last post on UAC, scopes are a pretty dry topic but critical for success with RealityServer. Hopefully this gives you a better understanding of what they are and how to apply them, but of course please contact us if you’d like to learn more.
Paul Arden has worked in the Computer Graphics industry for over 20 years, co-founding the architectural visualisation practice Luminova out of university before moving to mental images and NVIDIA to manage the Cloud-based rendering solution, RealityServer, now managed by migenius where Paul serves as CEO.