{"id":3613,"date":"2019-12-17T06:36:38","date_gmt":"2019-12-17T06:36:38","guid":{"rendered":"https:\/\/www.migenius.com\/?p=3613"},"modified":"2019-12-17T06:36:45","modified_gmt":"2019-12-17T06:36:45","slug":"whats-new-in-realityserver-5-3-update-276","status":"publish","type":"post","link":"https:\/\/www.migenius.com\/articles\/whats-new-in-realityserver-5-3-update-276","title":{"rendered":"What’s New in RealityServer 5.3 Update 276"},"content":{"rendered":"\n

This is the seventh update of RealityServer 5.3 since it was released in May. That’s a new release every month folks! While this post will focus on the features added in the most recent, 2593.276 build, we’ll also recap some of the other features added along the way in previous updates. The headline item for this new release is the ability to make HTTP requests from within the server-side V8 JavaScript API. This has been a long standing request from users and we are finally able to ship this feature to you. There’s also some cool new glTF features and much more.<\/p>\n\n\n\n\n\n\n\n

\"\"<\/figure>\n\n\n\n

HTTP Requests in V8<\/h3>\n\n\n\n

We’ve had this one in the backlog for quite a long time now. We build a lot of RealityServer applications for customers and there are so many times we’ve said to ourselves, if only we had HTTP request functionality we wouldn’t need to have this extra Node.js server.<\/p>\n\n\n\n

The recently added queue manager was borne of a similar frustration. We had implemented this functionality at least four times as a companion Node.js server application and started to see it should become a core feature. Afterwards we started to look at what other requirements caused us to have to use another server, just to get one key piece of functionality not available in RealityServer.<\/p>\n\n\n\n

The queue manager allowed us to lay the ground work since it required us to add a HTTP client to our C++ API (you can access this in our neuray Services API with the IHttp_client<\/a> class). After this was deployed we used this same C++ class to extend the V8 API with HTTP request functionality. We’ve tried to make this new API as convenient and easy to use as possible. It is as simple as adding something like this to your V8 command.<\/p>\n\n\n

\nconst response = http.get('http:\/\/www.example.com\/');\n<\/pre><\/div>\n\n\n
You’ll notice we are calling the get<\/em> method on our HTTP class here. This is just an alias which internally calls the send<\/em> method with the HTTP request method<\/em> property set to GET<\/em>. So this would be equivalent to.<\/p>\n\n\n
\nconst response = http.send({ method: 'GET', url: 'http:\/\/www.example.com\/' });\n<\/pre><\/div>\n\n\n
Similar methods are provided for the standard HTTP verbs GET<\/em>, POST<\/em>, DELETE<\/em>, HEAD<\/em>, OPTIONS<\/em>, PATCH<\/em> and PUT<\/em>. Of course the vast majority of the time get<\/em> and post<\/em> are the methods you’ll be using. Since a lot of web services use JSON for requests and responses you also have the option to have this automatically encoded and decoded for you just by setting the json<\/em> property of the request to true<\/em>.<\/p>\n\n\n
\nconst response = http.post({\n    url: 'http:\/\/api.example.com\/v1\/people',\n    json: true,\n    body: {\n        person_id: 'b4q5f3'\n    }\n});\nconsole.log(response.body.firstname);\n<\/pre><\/div>\n\n\n
In the above example we are sending a JSON encoded body and the response is automatically decoded from JSON into an object and set in the body<\/em> property of the response. This makes calling JSON based APIs from RealityServer really easy. There is also the ability to make multiple simultaneous requests, you just need to provide an array of URLs, request objects or a mixture of both. Here is a quick example.<\/p>\n\n\n
\nconst responses = http.send([\n    'http:\/\/example.com\/1',\n    'http:\/\/example.com\/2'\n]);\n<\/pre><\/div>\n\n\n
All of the requests will be sent simultaneously and the method will return only once all requests have completed. This is useful when downloading multiple resources if you want to make requests in parallel. There are a lot of other ways to use the API and many properties available on the request object to customise how your requests are sent. Please see the documentation for V8 Http Requests<\/a> for more details.<\/p>\n\n\n\n
Now, of course you may encounter errors when making HTTP requests. Note that responses that have HTTP error codes are not considered errors because they are valid responses, so if you need to react for example to a HTTP 404 code, then you need to check the code<\/em> property on the response for this. If however there is an error connecting to the server or another error that means the server did not actually provide a response then a http.Error<\/em> object will be returned instead of a http.Response<\/em> object. You can easily check for this like so.<\/p>\n\n\n
\nconst response = http.get('http:\/\/invaliddomain');\nif (response instanceof http.Error) {\n    throw response;\n}\n<\/pre><\/div>\n\n\n
Finally there is handling of binary data. If you are requesting binary data you need to ensure you set the encoding<\/em> property of your request to null<\/em>, otherwise the response will be interpreted as a utf-8 string. You will then get back an ArrayBuffer<\/em> as the body<\/em> property in the response. Let’s download a binary file and write it to disk.<\/p>\n\n\n
\nconst response = http.get({\n    url: 'http:\/\/example.com\/file.jpg',\n    encoding: null\n});\nfs.writeFile('file.jpg', response.body);\n<\/pre><\/div>\n\n\n
This is probably one of the more commonly requested reasons for having HTTP requests in V8, downloading content from other servers. Of course, you can also upload binary data to servers.<\/p>\n\n\n
\nconst image = fs.readFile('file.jpg');\nconst response = http.put({\n    url: 'http:\/\/example.com\/file.jpg',\n    body: image.buffer,\n    encoding: null\n});\n<\/pre><\/div>\n\n\n
These examples really just scratch the surface of what can be done. Of course once you have a general means of making HTTP requests you can connect virtually any service to RealityServer. Be sure to review the documentation<\/a> for the full details. We think this feature, like the new queue manager will greatly simplify the creation of RealityServer based applications.<\/p>\n\n\n\n
Assimp 5.0.0 and glTF 2.0<\/h3>\n\n\n\n
In this release we have updated the core Assimp<\/a> library to the newest 5.0.0 release (with a few additional patches we made which have been accepted by the project but not released yet). This brings enhancements to many of the file formats Assimp supports such as FBX, COLLADA and so on as well as a lot of bug fixes. See the Assimp 5.0.0 Release Notes<\/a> for full details of the changes since 4.1.0.<\/p>\n\n\n\n
\n
\n
This also brings some new glTF 2.0<\/a> features, including the two most requested by our users, support for the KHR_texture_transform<\/a> and KHR_lights_punctual<\/a> extensions. Texture transforms allow you do texture tiling independently of the models UV coordinates and also offset and rotate textures. Many glTF scenes are now using this functionality to remove the need for baking repeating textures into large atlases.<\/p>\n\n\n\n
 The KHR_lights_punctual<\/em> extension now results in lights of the specified type being created with valid values in RealityServer. Note however that right now these might not result in useful illumination without some further modifications (which of course you can do with our API). Also note that the range property is not supported since this is not physically correct and would cause problems during rendering. There are also very few example scenes available for this extension, so if you are using it and have good examples please contact us<\/a>. <\/p>\n<\/div>\n\n\n\n
\n
\"\"
glTF 2.0 Test Using KHR_texture_transform<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n
One other major feature we introduced (starting with build 2593.229) is new functionality to allow distilled MDL materials to be used in glTF 2.0 export (actually for simpler properties, other formats will use them as well). Right now this feature only works well in models where texture transformations are not used, however you can get some excellent results for scenes that support it. We’ve written up a separate post for this entitled Automatic Creation of glTF with Quality Materials<\/a>.<\/p>\n\n\n\n
Queue Manager<\/h3>\n\n\n\n
Queue manager functionality was added in build 2593.209 which allows you to utilise AWS SQS<\/a> as a job queue for running RealityServer commands. This is a major enhancement for customers doing more batch or job oriented workflows and also offers direct integration with AWS S3<\/a> so content can be directly uploaded back and served. You can now even combine this with the new HTTP request functionality to enable your V8 queue jobs to download content (for example pulling your models from S3) when running jobs. The queue manager is so important that we wrote up a dedicated blog post for it entitled Cost Effective Rendering with Amazon SQS and S3<\/a>. We’re also working on support for platforms other than AWS with Azure likely coming first. Get in touch<\/a> if you are looking for support for a specific platform other than AWS.<\/p>\n\n\n\n
Since releasing the queue manager we had a few requests for enhancements. It is now possible (actually mandatory) to specify a region for your SQS queue so you can have your queues outside the AWS us-east-1 region and we have also significantly expanded the options available to the S3 upload tasks to allow setting various metadata and options.<\/p>\n\n\n\n
Filling Out V8<\/h3>\n\n\n\n
We’ve been gradually filling in the gaps in our V8 API over the course of the year and we now have V8 wrapper classes for all major RealityServer element types. In general you can now build out your V8 commands without resorting to explicitly calling RealityServer commands with the RS.command<\/em> convention. When we build applications internally, every time we find ourselves needing to do this we add the wrapper class and as a result it has now gotten quite complete.<\/p>\n\n\n\n
This also includes some new additions to some of the core classes. For example we have now added fs.unlink<\/em> and fs.rmdir<\/em> to allow removing files and directories from within V8 (very handy when downloading things to disk using our new HTTP request functionality). With this latest release we are really starting to get to the point where there is much less need to resort to using other server technologies along side RealityServer in your application. While we don’t aim to replace full fledged application server platforms, we do want users to be able to accomplish the vast majority of RealityServer related tasks with just RealityServer.<\/p>\n\n\n\n
Mesh Splitting<\/h3>\n\n\n\n
This latest release also adds the new geometry_split_mesh<\/em> command which takes a Polygon_mesh<\/em> or Triangle_mesh<\/em> element and splits it into multiple meshes based on the material index assigned to each face. Unless you have a specific need this probably doesn’t sound very interesting but if you do need it, the command can save you a round trip back to your modelling software to separate the meshes by hand. Polygon meshes are automatically tessellated into triangle meshes for you during the splitting process.<\/p>\n\n\n\n
One of the reasons this command is being added is to test the initial mesh splitting support we have implemented which will go into a future release of our Assimp exporters. This will allow scenes using meshes with multiple materials to be exported to formats such as glTF which don’t support this, where as currently just the first material is exported. Even though we don’t have automatic support in the exporter yet, you can use the command to implement this already if needed. In a future release the exporter will handle this automatically.<\/p>\n\n\n\n
Smaller Additions<\/h3>\n\n\n\n
The mdl_is_default<\/em> command was added to allow you to determine if an MDL material instance or function call in the database is used as a default parameter in another material or function call. This is useful so you can exclude these elements which getting lists of materials and functions. To that end we also added a new include_defaults<\/em> parameter to the list_elements<\/em> command, if you set this to false (it defaults to true) then you won’t see such MDL elements when listing your elements out.<\/p>\n\n\n\n
We discovered during the course of some stress testing that the HTTP server had an internal hard coded limit of 100 concurrent connections. This isn’t generally as bad as it sounds since the operating system also plays a role in managing the traffic however Windows and Linux handle things differently and in general we wanted to be able to control this limit. There is now a http_concurrent_connection_limit<\/em> directive available in realityserver.conf<\/em> to control this.<\/p>\n\n\n\n
The image_get_original_filename<\/em> command (and the original_filename<\/em> property of the Image<\/em> class in V8) now give you back the filename that was originally given when loading an image. Before you could only get back the resolved filename (so this would be where the file was actually found on the local file system). There was a few rare use cases that popped up where the original filename was needed so we added this command.<\/p>\n\n\n\n
Iray Updates<\/h3>\n\n\n\n
This version and the previous build are using Iray RTX 2019.1.5 build 317500.7473. Of course there are quite a few fixes and updates made to Iray which we incorporate so you should always check the neurayrelnotes.pdf<\/em> file that comes with RealityServer to see what has changed. If you are using RTX based hardware it is particularly important to keep up with the latest releases of RealityServer since the Iray updates often bring performance improvements.<\/p>\n\n\n\n
What’s Next?<\/h3>\n\n\n\n
Well, we’d like to hear from you. What would you like to see added to RealityServer next? Almost all features added over the last year have been a direct result of user feedback, or because we kept implementing the same thing for different customers over and over. Reach out<\/a> with your ideas, whether your a long time user or thinking RealityServer would fit your needs, if only it did that one extra thing, we want to hear from you.<\/p>\n","protected":false},"excerpt":{"rendered":"
This is the seventh update of RealityServer 5.3 since it was released in May. That’s a new release every month folks! While this post will focus on the features added in the most recent, 2593.276 build, we’ll also recap some of the other features added along the way in previous updates. The headline item for […]<\/p>\n","protected":false},"author":2,"featured_media":3620,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"spay_email":"","jetpack_publicize_message":"Checkout what's New in RealityServer 5.3 Update 276. Good news for those waiting for V8 HTTP request and glTF KHR_texture_transform support.","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[15],"tags":[14],"jetpack_featured_media_url":"https:\/\/www.migenius.com\/migenius\/wp-content\/uploads\/2019\/12\/rs_53_update1_feature2.jpg","_links":{"self":[{"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/posts\/3613"}],"collection":[{"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/comments?post=3613"}],"version-history":[{"count":5,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/posts\/3613\/revisions"}],"predecessor-version":[{"id":3621,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/posts\/3613\/revisions\/3621"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/media\/3620"}],"wp:attachment":[{"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/media?parent=3613"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/categories?post=3613"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.migenius.com\/wp-json\/wp\/v2\/tags?post=3613"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}