Implementation of an importer

This topic introduces:

Basic concepts about the implementation and usage of custom importers used in conjunction with the Iray API:
- Implementing an importer
- Registering an importer
An example importer program consisting of vanilla_importer.h and example_importer.cpp. The importer is called Vanilla. For simplicity, it is defined in and used by the main application. The importer is intended as an illustrative skeleton: It implements all interfaces but does not parse the file content in a meaningful way.

Implementing an importer

The implementation of the Vanilla importer is structured in three parts:

Implementing the mi::IImpexp_state interface
Implementing the mi::neuraylib::IImporter interface
Implementing the mi::neuraylib::IImporter::import_elements() method

Instances of mi::IImpexp_state are used to pass information about the current importer state, for example to recursive calls of importers. The Vanilla importer does not need to carry around any additional information besides what is required by the interface, therefore this simple implementation is fine. Note that the simple derivation from mi::base::Interface_implement suffices here (in contrast to example_plugins.cpp).

The Vanilla importer is given in the implementation of the mi::neuraylib::IImporter interface. Later, an instance of this class is registered as an importer with the Iray API. Most of the methods implemented here are defined in the base interface mi::IImpexp_state, as they are common for importers and exporters. The Vanilla importer claims to handle files with the extension .vnl and .van. It does not require specific capabilities of the reader to handle these formats. However, if the reader supports the lookahead capability, it will use a magic header check instead of relying on file name extensions.

The actual work of the Vanilla importer occurs in the import_elements() method. It is split into three parts:

Creating an import result object
Creating a group object which acts as the root group
Reading the file line by line

While performing these tasks, the example program demonstrates what type of errors to detect, how errors can be reported, and how to implement an include file mechanism, and similar things, with a recursive call to the mi::neuraylib::IImport_api::import_elements() method.

Registering an importer

Registering importers is similar to registering user-defined classes. However, since importers are different from regular classes (for example, you cannot create instances of them using mi::neuraylib::ITransaction::create()); you need to use a registration method specific to importers. This registration method expects a pointer to an instance of the custom importer.

To run the example, you need to create two files called test1.vnl and test2.van, each five lines long and starting with a line saying VANILLA. For demonstration purposes, the example will print the generated error messages and the names of the imported elements.

vanilla_importer.h

Plain text

001 /******************************************************************************
002  * © 1986, 2014 NVIDIA Corporation. All rights reserved.
003  *****************************************************************************/
004 
005 #include <mi/neuraylib.h>
006 
007 #include <string>
008 #include <sstream>
009 
010 // Support function to handle mi::base::Message_severity to std::string conversion
011 static std::string enum_to_str( mi::base::Message_severity severity)
012 {
013     switch( severity) {
014         case mi::base::MESSAGE_SEVERITY_FATAL:
015             return "fatal";
016         case mi::base::MESSAGE_SEVERITY_ERROR:
017             return "error";
018         case mi::base::MESSAGE_SEVERITY_WARNING:
019             return "warning";
020         case mi::base::MESSAGE_SEVERITY_INFO:
021             return "information";
022         case mi::base::MESSAGE_SEVERITY_VERBOSE:
023             return "verbose";
024         case mi::base::MESSAGE_SEVERITY_DEBUG:
025             return "debug";
026         default:
027             return "unknown" ;
028     }
029 }
030 
031 // Define importer state, which is used in the importer function to carry additional data around
032 // for possible recursive invocations of importers. It contains a URI for the imported resource,
033 // a line number, and a pointer to a parent state supporting recursive imports.
034 class Vanilla_import_state
035     : public mi::base::Interface_implement< mi::IImpexp_state>
036 {
037     // Member variables to keep all necessary data
038     std::string               m_uri;
039     mi::Uint32                m_line_number;
040     const mi::IImpexp_state*  m_parent_state;
041 
042 public:
043     // Definition of all interface functions.
044 
045     const char* get_uri() const { return m_uri.empty() ? 0 : m_uri.c_str(); }
046 
047     mi::Uint32 get_line_number() const { return m_line_number; }
048 
049     void set_line_number( mi::Uint32 number) { m_line_number = number; }
050 
051     void incr_line_number() { ++m_line_number; }
052 
053     const mi::IImpexp_state* get_parent_state() const { return m_parent_state; }
054 
055     // Definition of corresponding setters / constructors. They are not part of the interface.
056 
057     // Default constructor, initializes line number to 1.
058     Vanilla_import_state()
059         : m_line_number( 1),
060           m_parent_state( 0)
061     {}
062 
063     void set_uri( const char* uri) { m_uri = uri ? uri : ""; }
064 
065     void set_parent_state( const mi::IImpexp_state* parent_state)
066     {
067         m_parent_state = parent_state;
068     }
069 };
070 
071 // Define importer. It defines all meta information, for example, author, version numbers,
072 // which formats it supports etc. The longer format detection functions and import_elements
073 // function are implemented outside of the class body.
074 class Vanilla_importer
075     : public mi::base::Interface_implement< mi::neuraylib::IImporter>
076 {
077     mi::neuraylib::IPlugin_api* m_iplugin_api;
078 
079 public:
080     // Returns a state suitable for passing it to an import call.
081     // The parameters are used to initialize the corresponding properties of the state.
082     // The line number is set to 1.
083     mi::IImpexp_state* create_impexp_state (
084         const char* uri,
085         const mi::IImpexp_state* parent_state) const
086     {
087         Vanilla_import_state* import_state = new Vanilla_import_state();
088         import_state->set_uri( uri);
089         import_state->set_parent_state( parent_state);
090         return import_state;
091     }
092 
093     // This importer supports the file name extensions ".vnl" and ".van".
094     const char* get_supported_extensions( mi::Uint32 i) const
095     {
096         switch( i) {
097             case 0:  return ".vnl";
098             case 1:  return ".van";
099             default: return 0;
100         }
101     }
102 
103     // Returns the confidence of the importer that its test_file_type() can identify the file and
104     // that the file format is fully supported.
105     mi::Impexp_priority get_priority () const
106     {
107         return mi::IMPEXP_PRIORITY_WELL_DEFINED;
108     }
109 
110     // Returns a concise single-line clear text description of the importer.
111     const char* get_name () const
112     {
113         return "mental images example vanilla (v1) importer";
114     }
115 
116     // Returns a concise single-line clear text description of the author of
117     // this importer.
118     const char* get_author () const
119     {
120         return "mental images GmbH, Berlin, Germany";
121     }
122 
123     // Returns the unique identifier for the importer.
124     mi::base::Uuid get_uuid() const
125     {
126         mi::base::Uuid uuid;
127         uuid.m_id1 = 0x338eca60;
128         uuid.m_id2 = 0x31004802;
129         uuid.m_id3 = 0xaab9046b;
130         uuid.m_id4 = 0x9e0b1d9b;
131         return uuid;
132     }
133 
134     // Returns the major version number of the importer.
135     mi::Uint32 get_major_version() const { return 1; }
136 
137     // Returns the minor version number of the importer.
138     mi::Uint32 get_minor_version() const { return 0; }
139 
140     // Returns true if the importer can handle the file type determined by the file name extension.
141     bool test_file_type ( const char* extension) const;
142 
143     // Returns true if the importer can handle the file type determined by the file name extension
144     // and if the reader has sufficient capabilities for import.
145     bool test_file_type( const char* extension,
146                                  const mi::IReader* reader) const;
147 
148     // Imports all elements from the reader in a format determined by the file extension and
149     // (optionally) the lookahead of the reader.
150     mi::IImport_result* import_elements (
151         mi::neuraylib::ITransaction* transaction,
152         const char*                  extension,
153         mi::IReader*                 reader,
154         const mi::IMap*              options,
155         mi::IImpexp_state*           import_state) const;
156 
157     // Definition of constructors and support functions. They are not part of the interface.
158 
159     // Constructor.
160     Vanilla_importer( mi::neuraylib::IPlugin_api* iplugin_api)
161         : m_iplugin_api( iplugin_api)
162     {
163         m_iplugin_api->retain();
164     }
165 
166     // Destructor.
167     ~Vanilla_importer()
168     {
169         m_iplugin_api->release();
170     }
171 
172     // Format message with context and append it to the messages in the result.
173     static mi::IImport_result_ext* report_message(
174         mi::neuraylib::ITransaction* /*transaction*/,
175         mi::IImport_result_ext*      result,
176         mi::Sint32                   message_number,
177         mi::base::Message_severity   message_severity,
178         std::string                  message,
179         const mi::IImpexp_state*     import_state) // not 0
180     {
181         std::ostringstream s;
182         s << import_state->get_uri()
183           << ":" << import_state->get_line_number() << ": "
184           << "Vanilla importer message " << message_number << ", "
185           << "severity " << enum_to_str( message_severity) << ": "
186           << message;
187         // Report context of all parent import states from recursive
188         // invocations of import_elements in their own lines with indentation.
189         import_state = import_state->get_parent_state();
190         while( import_state) {
191             s << "\n    included from: " << import_state->get_uri()
192               << ":" << import_state->get_line_number();
193             import_state = import_state->get_parent_state();
194         }
195         result->message_push_back( message_number, message_severity, s.str().c_str());
196         return result;
197     }
198 };
199 
200 // Returns true if the importer can handle the file type determined by the file name extension.
201 bool Vanilla_importer::test_file_type ( const char* extension ) const
202 {
203     // This importer supports the file name extensions ".vnl" and ".van".
204     mi::Size len = std::strlen( extension);
205     return (len > 3)
206         &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
207           || ( 0 == strcmp( extension + len - 4, ".van")));
208 }
209 
210 // Returns true if the importer can handle the file type determined by the file name extension
211 // and if the reader has sufficient capabilities for import.
212 bool Vanilla_importer::test_file_type( const char* extension,
213                                        const mi::IReader* reader ) const
214 {
215     // Use magic header check if lookahead is available
216     if ( reader->supports_lookahead()) {
217         // File has to start with "VANILLA" and linebreak, which can
218         // be \n or \r depending on the line ending convention in the file.
219         const char** buffer = 0;
220         mi::Sint64 n = reader->lookahead( 8, buffer);
221         return ( n >= 8) && (0 == std::strncmp( *buffer, "VANILLA", 7))
222                          && ((*buffer[7] == '\n') || (*buffer[7] == '\r'));
223     }
224     // This importer supports the file name extensions ".vnl" and ".van".
225     mi::Size len = std::strlen( extension);
226     return (len > 3)
227         &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
228           || ( 0 == strcmp( extension + len - 4, ".van")));
229 }
230 
231 // Imports all elements from the reader in a format determined by the file extension and
232 // (optionally) the lookahead of the reader.
233 mi::IImport_result* Vanilla_importer::import_elements (
234     mi::neuraylib::ITransaction* transaction,
235     const char*                  extension,
236     mi::IReader*                 reader,
237     const mi::IMap*              importer_options,
238     mi::IImpexp_state*           import_state) const
239 {
240     // Create the importer result instance for the return value.
241     // If that fails something is really wrong and we return 0.
242     mi::IImport_result_ext* result
243         = transaction->create<mi::IImport_result_ext>( "Import_result_ext");
244     if( !result)
245         return 0;
246 
247     // Get the 'prefix' option.
248     MISTD::string prefix;
249     if( importer_options && importer_options->has_key( "prefix")) {
250         mi::base::Handle<const mi::IString> option(
251             importer_options->get_value<mi::IString>( "prefix"));
252         prefix = option->get_c_str();
253     }
254 
255     // Get the 'list_elements' option.
256     bool list_elements = false;
257     if( importer_options && importer_options->has_key( "list_elements")) {
258         mi::base::Handle<const mi::IBoolean> option(
259             importer_options->get_value<mi::IBoolean>( "list_elements"));
260         list_elements = option->get_value<bool>();
261     }
262 
263     // Before we start parsing the file, we create a group that collects all
264     // top-level elements. This will be our rootgroup.
265     std::string root_group_name = prefix + "Vanilla::root_group";
266     mi::base::Handle<mi::IGroup> rootgroup( transaction->create<mi::IGroup>( "Group"));
267     mi::Sint32 error_code = transaction->store( rootgroup.get(), root_group_name.c_str());
268     if( error_code != 0)
269         return report_message( transaction, result, 4010, mi::base::MESSAGE_SEVERITY_ERROR,
270             "failed to create the root group", import_state);
271     rootgroup = transaction->edit<mi::IGroup>( root_group_name.c_str());
272 
273     // Register the rootgroup with the importer result.
274     result->set_rootgroup( root_group_name.c_str());
275 
276     // If the element list flag is set, record the rootgroup element also in the
277     // elements array of the result.
278     if ( list_elements)
279         result->element_push_back( root_group_name.c_str());
280 
281     // Assume it is a line based text format and read it line by line.
282     // Assume lines are no longer than 256 chars, otherwise read it in pieces
283     char buffer[257];
284     while ( reader->readline( buffer, 257) && buffer[0] != '\0') {
285 
286         // Periodically check whether the transaction is still open. If not, stop importing.
287         if( !transaction->is_open())
288             break;
289 
290         // Here you can process the buffer content of size len.
291         // It corresponds to the line import_state->get_line_number() of the input file.
292         mi::Size len = std::strlen( buffer);
293 
294         // We illustrate some actions triggered by fixed line numbers:
295         // Line 3 of a ".vnl" file triggers the recursive inclusion of the file "test2.van" file
296         // with a prefix.
297         mi::Size ext_len = std::strlen( extension);
298         if ( 3 == import_state->get_line_number()
299              && (ext_len > 3)
300              && 0 == strcmp( extension + ext_len - 4, ".vnl")) {
301             // Get a IImport_api handle to call its import_elements() function
302             mi::base::Handle<mi::neuraylib::IImport_api> import_api(
303                 m_iplugin_api->get_api_component<mi::neuraylib::IImport_api>());
304             if ( ! import_api.is_valid_interface())
305                 // Error numbers from 4000 to 5999 are reserved for custom importer messages like
306                 // this one
307                 return report_message( transaction, result,
308                    4001, mi::base::MESSAGE_SEVERITY_ERROR,
309                    "did not get a valid IImport_api object, import failed", import_state);
310             // Call the importer recursively to illustrate the handling of include files and
311             // similar things. We trigger this import only on a ".vnl" file and include the fixed
312             // file "test2.van". We give the included file an extra prefix "Prefix_".
313             mi::base::Handle<mi::neuraylib::IFactory> factory(
314                 m_iplugin_api->get_api_component<mi::neuraylib::IFactory>());
315             mi::base::Handle<mi::IString> child_prefix( factory->create<mi::IString>( "String"));
316             child_prefix->set_c_str( "Prefix_");
317             mi::base::Handle<mi::IMap> child_importer_options( factory->clone<mi::IMap>(
318                 importer_options));
319             child_importer_options->erase( "prefix");
320             child_importer_options->insert( "prefix", child_prefix.get());
321             mi::base::Handle<const mi::IImport_result> include_result(
322                 import_api->import_elements( transaction, "file:test2.van",
323                     child_importer_options.get(), import_state));
324             // Safety check, if this fails, the import is not continued.
325             if ( ! include_result.is_valid_interface())
326                 return report_message( transaction, result,
327                    4002, mi::base::MESSAGE_SEVERITY_ERROR,
328                    "import was not able to create result object, import failed", import_state);
329             // Process the result. Even in the case of an error, we need to process the
330             // elements array.
331             if ( list_elements)
332                 result->append_elements( include_result.get());
333             // Append messages of include to this result
334             result->append_messages( include_result.get());
335             // React on errors during processing of the include
336             if ( include_result->get_error_number() > 0) {
337                 // Report the failure of the include as a separate message too
338                 report_message( transaction, result,
339                     4003, mi::base::MESSAGE_SEVERITY_ERROR,
340                     "including file 'test2.van' failed.", import_state);
341             } else {
342                 // Recursive import was successful. The rootgroup of the
343                 // import is now appended to this rootgroup
344                 if ( 0 == include_result->get_rootgroup())
345                     report_message( transaction, result,
346                         4004, mi::base::MESSAGE_SEVERITY_ERROR,
347                         "include file 'test2.van' did not contain a rootgroup", import_state);
348                 else
349                     rootgroup->attach( include_result->get_rootgroup());
350             }
351         }
352 
353         // Line 4 triggers several messages and adds an empty group to the rootgroup.
354         if ( 4 == import_state->get_line_number()) {
355             // Several messages, file parsing continues
356             report_message( transaction, result, 4005, mi::base::MESSAGE_SEVERITY_FATAL,
357                           "test message in line 4", import_state);
358             report_message( transaction, result, 4006, mi::base::MESSAGE_SEVERITY_ERROR,
359                           "test message in line 4", import_state);
360             report_message( transaction, result, 4007, mi::base::MESSAGE_SEVERITY_WARNING,
361                           "test message in line 4", import_state);
362             report_message( transaction, result, 4008, mi::base::MESSAGE_SEVERITY_INFO,
363                           "test message in line 4", import_state);
364             report_message( transaction, result, 4009, mi::base::MESSAGE_SEVERITY_VERBOSE,
365                           "test message in line 4", import_state);
366             // Create a group "Vanilla::Group1"
367             std::string group_name = prefix + "Vanilla::Group1";
368             mi::base::Handle<mi::IGroup> group( transaction->create<mi::IGroup>( "Group"));
369             mi::Sint32 error_code = transaction->store( group.get(), group_name.c_str());
370             if( error_code != 0)
371                 report_message( transaction, result, 4011, mi::base::MESSAGE_SEVERITY_ERROR,
372                     "unexpected error in line 4", import_state);
373             else {
374                 // Add this group to the rootgroup
375                 rootgroup->attach( group_name.c_str());
376                 // If get_list_elements_flag is set, record the new element in the elements array
377                 // of the result.
378                 if ( list_elements)
379                     result->element_push_back( group_name.c_str());
380             }
381         }
382 
383         // Handle line numbers, buffer might end in '\n' or not if line was too long.
384         if ((len > 0) && ('\n' == buffer[len-1]))
385             import_state->incr_line_number();
386     }
387     if ( reader->eof()) {
388         // Normal end
389         return result;
390     }
391     // Report error condition for a failed reader call
392     return report_message( transaction,
393                          result,
394                          reader->get_error_number(),
395                          mi::base::MESSAGE_SEVERITY_ERROR,
396                          reader->get_error_message() ? reader->get_error_message() : "",
397                          import_state);
398 }

example_importer.cpp