Sdk

[dcnieho]

Dear Worldviz,

I've already sent this email to support before I noticed the dedicated forum, I guess this is the best place to ask.

I'd like to create a custom render plugin with Vizard 4.0. Given the updated OSG headers and such required, could you provide me with the 4.0 SDK (even if its only ready for a private test)? Also, please let me know if there will be any documentation updates regarding such plugins versus the 3.0 docs.

Thanks!
Dee

[farshizzo]

The latest OSG header/lib files are now available in the Vizard 4.0 SDK download on the Vizard download page

[dcnieho]

@Farshizzo
thanks! The exaple customnode plugin compiled out of the box.

Could you add a note in the documentation though that it must be compiled with VS2008? It puzzled me until i was lucky to find that info on the forum.

In general, a more complete example (e.g. showing how to send command that change the object drawn by thw plugin) would be really helpful and useful!

Thanks,
Dee

[TarkaDahl]

Hi

Thanks for telling us that it can be compiled in vs2008, ill get a copy of that this afternoon(if i can find a copy).

Ive only got to recompile animpath.dlm and dldraw.dlr (if anyone has compiled any of these can they post them). Everything else works fine. The new interface is amazing. Im finding the inspector to be really useful aswell.

The only problem i had, was when debugging i accidently shut down the local/global window and couldnt get it back with out restarting vizard. Maybe it was me being stupid.

Thanks

TarkaDahl

[farshizzo]

Sorry I forgot to mention that. You definitely need to compile the plugins using Visual Studio 2008.

We're hoping to finalize the new plugin interface before the final release. When that's ready there will be a more complete example of how to interact with your plugin objects from the Python script. The old style plugins will still be compatible though.

[farshizzo]

Originally Posted by TarkaDahl

The only problem i had, was when debugging i accidently shut down the local/global window and couldnt get it back with out restarting vizard. Maybe it was me being stupid.

The debug panes (Call Stack, Locals, Globals) are automatically made visible when debugging starts and hidden when debugging ends. If you accidentally close the pane, then restarting the debugger should bring it back up. We'll add some options in the View menu to allow manually bringing the windows back up, in case this happens.

[dcnieho]

Originally Posted by farshizzo

We're hoping to finalize the new plugin interface before the final release. When that's ready there will be a more complete example of how to interact with your plugin objects from the Python script.

Ah, such a more complete example will definately be very welcome. I've just been experimenting with adding my own functions to the CustomDrawable object in the example code (my final aim is to provide to Vizard a rather extensive rendering library I normally use with a GLUT based program, so all I want to do is write a interface/wrapper).

This came out to be rather complicated Am I doing this right? The below code shows the growing exprression (i send the pointers to cout to still if they're still valid or I've made a mistake).

cout << ((VizCustomNodeObj*)custom)->node << endl;
cout << osg::dynamic_pointer_cast<osg::Geode>(((VizCustomNodeObj*)custom)->node) << endl;
cout << osg::dynamic_pointer_cast<osg::Geode>(((VizCustomNodeObj*)custom)->node)->getDrawableList()[0] << endl;
cout << osg::dynamic_pointer_cast<CustomDrawable>(osg::dynamic_pointer_cast<osg::Geode>(((VizCustomNodeObj*)custom)->node)->getDrawableList()[0]) << endl;
osg::dynamic_pointer_cast<CustomDrawable>(osg::dynamic_pointer_cast<osg::Geode>(((VizCustomNodeObj*)custom)->node)->getDrawableList()[0])->ToggleAnimate();

I can now toggle whether the example object is stretched or not :P

Best,
Dee

[dcnieho]

Oh, I have another question:
How do I communicate fatal, execution-halting errors to Vizard from my plugin? As there's C code in between, I can't throw exceptions (right?), so how can I make Vizard stop execution from my plugin then (I know I can send info to stdout from the plugin, so I don't need a way to send a message along as well, just the execution halt)?

Thanks!
Dee

[dcnieho]

Ah, another thing I hope you could add to the documentation:

The plugin is compiled with _SECURE_SCL=0. This means that if you call functions in libraries that also take stl containers, those libraries also have to be compiled with preprocessro definition set. Otherwise, vectro has a different size and you get difficult to trace errors (I just spent a few hours on this)...

Best,
Dee

[dcnieho]

hmm, actually, having "_SECURE_SCL=0" set is kind of disabling as its not default. Is this temporary until some kinks are worked out? Even if your plugins are compiled with this switch by default it should be possible to set "_SECURE_SCL=1" without the code dying on me

---------------------------
Microsoft Visual C++ Runtime Library
---------------------------
Runtime Error!

Program: C:\Program Files\WorldViz\Vizard4\bin\winviz.exe

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

---------------------------
OK   
---------------------------

[farshizzo]

You will need to add "_SECURE_SCL=0" to the preprocessor definition list of your project. The example projects should have this defined.

Here is a list of Visual Studio 2008 project settings you should set for all your plugins:

C/C++:
    Preprocessor:
        Preprocessor Definitions += "_SECURE_SCL=0"
    Code Generation:
        Runtime Library = Multi-threaded DLL

Linker:
    Manifest File:
        Generate Manifest = No

[farshizzo]

A new beta has been released, which supports the new plugin interface. The Vizard 4.0 SDK on our download page contains a new sample project showing how to use the new interface.

Let us know if you have any problems or suggestions.

[dcnieho]

Hi farshizzo,

Just noted your message, thank you!

Am I correct to understand that an extension is an overall plugin wrapper that could contain custom nodes, sensors and textures? So I can simply use the node only if that's all I need? From what I'm seeing, I'm liking the syntax, thank you!

And another question to make sure I'm doing things correctly: If I prefer to define the python interface code in a separate file instead of in the C file. for that, do I need to delete everything in between PYTHON_BEGIN_RUN_CODE and PYTHON_END_RUN_CODE in VizExtension* CreateVizardExtension(viz:ata &data)?

Thank you!

Best,
Dee

[dcnieho]

Dear Farshizzo,

Looking at the code more (and the C++ implementation of viz:ata), is it true that it is possible to pass an arbitrary number of arguments (with arbitrary names and somewhat arbitrary datatypes) through the Python interface to the C++ extension code? That's great! But how would one go about that at the Python side? Hmm, looking at the code in viz.VizExtensionNode and viz.VizExtension maybe its not possible yet). Anyway, that would be very cool! I do see I am able to pass back to Python arbitrary data structures (because lists and such are supported! and with some hacking any other Python datatype i see)

Another question is how to set error conditions in the C++ code and communicate them to the Python code? In the old code, one could say ((VizCustomNodeObj*)custom)->status = 0; in some functions (but not CommandCustomNode sadly) which would cause Vizard to communicate an error ocurred. Is it possible through the interface defined in pyerrors.h? If something is possible, could you add an example?

Lastly, I'm happy to see the void MyExtension::update(const viz::Event &e) function, that seems useful! Some questions though:
1) From inside this function, is it possible to retrieve pointers to the Nodes/Sensors/Textures attached to the extension to perform operations on them and/or pass data along?
2) I see that the viz::Event object has function for getting the X, Y and Z input arguments, amongst others. How are these set?
3) related to that, is it possible to override the update call from Python? I'd like to define my own function to run every frame (I now do that using vizact.ontimer(0,self.stim.update), where self.stim.update is a function i defined) where it does some processing and then passes some data along to the plugin. If you answer to 1) is no, I guess I'll keep doing what I'm doing now?

Thank you for the answers!
Best,
Dee

[dcnieho]

Dear Farshizzo,

I did run into some trouble. My C++ also uses boost, and I get a whole bunch of redefinition warnings for boost::any.

My version of boost (as does the current trunk version) apparently defines #define BOOST_ANY_INCLUDED instead of #define BOOST_ANY_HPP, leading to this situation.

Could you also add the other define to your custom header or is there a more appropriate way for me or you to deal with this?

Best,
Dee

[dcnieho]

Oh, turns out I had to solve it by changing the header of MyNode.h to:

#include <boost/any.hpp>
#define BOOST_ANY_HPP // fool Vizard's boost::any
#include <VizExtensionOSGNode.h>

Maybe the best solution is for your any to life in a different namespace? Then mixing different versions of boost will be just fine as your any will only affect your code (its header only so it shouldn't lead to any linking trouble).

EDIT:
urgh, spoke too fast, compilation wasn't finished yet. Turns out even this doesn't solve things:
1>MyExtension.cpp
1>c:\dat\C\projects\Dependency\boost\boost/any.hpp(36) : error C2011: 'boost::any' : 'struct' type redefinition
1> c:\dat\C\projects\Dependency\Vizard4_SDK\include\v iz/any(230) : see declaration of 'boost::any'
1>c:\dat\C\projects\Dependency\boost\boost/any.hpp(165) : error C2011: 'boost::bad_any_cast' : 'struct' type redefinition
1> c:\dat\C\projects\Dependency\Vizard4_SDK\include\v iz/any(32) : see declaration of 'boost::bad_any_cast'
1>c:\dat\C\projects\Dependency\boost\boost/any.hpp(186) : error C2995: 'T *boost::any_cast(boost::any *)' : function template has already been defined
1> c:\dat\C\projects\Dependency\Vizard4_SDK\include\v iz/any(390) : see declaration of 'boost::any_cast'
1>c:\dat\C\projects\Dependency\boost\boost/any.hpp(192) : error C2995: 'const T *boost::any_cast(const boost::any *)' : function template has already been defined

Best,
Dee

[dcnieho]

Ok, I have removed my code portion using boost:any (wasn't used in the case of this plugin), so that fixed that problem.

Switching over to the new interface was easy and things seem to work well, nice work!

I am having one problem that I didn't have with the old code: when compiling the plugin in debug mode it doesn't work. Specifically, i get an access violation somewhere deep in xtree on the "int command = data.getInt("command");" call in viz::Referenced* MyExtension::createNode(viz:ata &data) (at DNStimuliExtension_d.dle!std::_Tree<std::_Tmap_tra its<std::basic_string<char,std::char_traits<char>, std::allocator<char> >,boost::any,std::less<std::basic_string<char,std: :char_traits<char>,std::allocator<char> > >,std::allocator<std:air<std::basic_string<char, std::char_traits<char>,std::allocator<char> > const ,boost::any> >,0> >::_Lbound(const std::basic_string<char,std::char_traits<char>,std: :allocator<char> > & _Keyval="command") Line 1264 + 0x8 bytes C++
to be exact, the _Mysize property of the tree object is set to 268525969 btw!). For this it doesn't matter if I define NDEBUG or _DEBUG in the preprocessor directives, it must be some other difference between the two modes that creates the trouble. I'd like to be able to compile a debug mode plugin (I was able to add debug information and everything as long as I kept NDEBUG defined instead of _DEBUG in the old interface) so I can step through meaningful code from visualstudio upon a crash in the plugin (even when run from Vizard, Visual studio pops up a window automatically upon a crash asking me how to debug, it worked very well!).

Is there any way you could support this? Oh yeah, btw, as python27_d.lib isn't shipped with the SDK, I simply created a copy of python27.lib with that name, but I don't think that would be the problem?

Best!
Dee

[farshizzo]

Originally Posted by dcnieho

And another question to make sure I'm doing things correctly: If I prefer to define the python interface code in a separate file instead of in the C file. for that, do I need to delete everything in between PYTHON_BEGIN_RUN_CODE and PYTHON_END_RUN_CODE in VizExtension* CreateVizardExtension(viz:ata &data)?

Defining the extension interface is optional, so removing the python code should not be an issue.

[farshizzo]

Originally Posted by dcnieho

Looking at the code more (and the C++ implementation of viz::Data), is it true that it is possible to pass an arbitrary number of arguments (with arbitrary names and somewhat arbitrary datatypes) through the Python interface to the C++ extension code? That's great! But how would one go about that at the Python side? Hmm, looking at the code in viz.VizExtensionNode and viz.VizExtension maybe its not possible yet). Anyway, that would be very cool! I do see I am able to pass back to Python arbitrary data structures (because lists and such are supported! and with some hacking any other Python datatype i see)

That was the idea with the viz::Data structure. Currently you are limited to passing the fixed arguments through the Python interface though. Eventually we would like to expose the ability to pass more complex data types through Python.

Originally Posted by dcnieho

Another question is how to set error conditions in the C++ code and communicate them to the Python code? In the old code, one could say ((VizCustomNodeObj*)custom)->status = 0; in some functions (but not CommandCustomNode sadly) which would cause Vizard to communicate an error ocurred. Is it possible through the interface defined in pyerrors.h? If something is possible, could you add an example?

The new interface allows you to return arbitrary Python objects from the command method, so you can return anything you wish to signal an error. If you want to raise a Python exception from your plugin, then you can use the Python PyErr_SetString function to raise an exception.

Originally Posted by dcnieho

1) From inside this function, is it possible to retrieve pointers to the Nodes/Sensors/Textures attached to the extension to perform operations on them and/or pass data along?

No, but you can internally save a reference to them. This is what most of the built-in plugins do. For example, the extension class can keep a list of sensor objects created by it:

std::list< viz::ref_ptr< MySensor > > m_sensors;

Originally Posted by dcnieho

2) I see that the viz::Event object has function for getting the X, Y and Z input arguments, amongst others. How are these set?

The viz::Event object passed to the plugins only provides the frame information described in the docs. The X, Y, Z arguments are not used for update events.

Originally Posted by dcnieho

3) related to that, is it possible to override the update call from Python? I'd like to define my own function to run every frame (I now do that using vizact.ontimer(0,self.stim.update), where self.stim.update is a function i defined) where it does some processing and then passes some data along to the plugin. If you answer to 1) is no, I guess I'll keep doing what I'm doing now?

If you want Python code to run every frame, then you can use the existing Python methods of registering an update function. You can also call methods of the extensions Python interface from within the C++ code. For example, if you added a method called myupdate to the extension Python interface, then the following code could be used to call it:

PYTHON_BEGIN_RUN_CODE
  PyObject *pyExt = Python_GetExtensionObject(GetID());
  if(pyExt) {
    PyObject *pyReturn = PyObject_CallMethod(pyExt,"myupdate");
    PYTHON_FLUSHERROR;
    Py_XDECREF(pyReturn);
    Py_DECREF(pyExt);
  }
PYTHON_END_RUN_CODE

[farshizzo]

Originally Posted by dcnieho

Could you also add the other define to your custom header or is there a more appropriate way for me or you to deal with this?

Thanks, we'll modify the included header so it doesn't conflict with boost.

[farshizzo]

Originally Posted by dcnieho

Is there any way you could support this? Oh yeah, btw, as python27_d.lib isn't shipped with the SDK, I simply created a copy of python27.lib with that name, but I don't think that would be the problem?

You need to compile your code in Release mode. Mixing debug/release binaries can cause problems, especially when using the stl.

[dcnieho]

Hi Farshizzo,

Thank you for all the detailed information! I still have some questions though:

Originally Posted by farshizzo

No, but you can internally save a reference to them. This is what most of the built-in plugins do. For example, the extension class can keep a list of sensor objects created by it:

std::list< viz::ref_ptr< MySensor > > m_sensors;

Is this supposed to be

std::list< osg::ref_ptr< MySensor > > m_sensors;

instead? I don't seem to have a viz::ref_ptr (but do have a viz::Referenced*). In any case, things seem to work fine with osg::ref_ptr

Originally Posted by farshizzo

The new interface allows you to return arbitrary Python objects from the command method, so you can return anything you wish to signal an error. If you want to raise a Python exception from your plugin, then you can use the Python PyErr_SetString function to raise an exception.

I am trying the Python PyErr_SetString method and I'm having some trouble with it. If I do a normal (non-debug) run and raise/set an error in my plugin, an error occurs on the next command call in the python node class that interfaces with the C++ node instance:

Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "C:\dat\C\projects\Vizard dll\testCustomNode.py", line 58, in <module>
    stim.setStimSpec('GDcl')
  File "DNstimuli.py", line 45, in setStimSpec
    self.command(1,stimSpec)
  File "C:\Program Files\WorldViz\Vizard4\python\viz.py", line 6837, in command
    return _ipcSend(_VIZ_CMDEXTNODE,self.id,command,mesg,x,y,z,w)
TypeError: command must be an int

The line self.command(1,stimSpec) is the first command I sent to the node after the command in which the exception was raised (which created no output or anything when raised), there can be any amount of code in between these two calls.

If setup a debug breakpoint on this first call after the raised exception (self.command(1,stimSpec)), and try to step into it, i get this output:

C:\Program Files\WorldViz\Vizard4\python\vizbdb.py:410: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  if id in ids:
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "C:\dat\C\projects\Vizard dll\testCustomNode.py", line 58, in <module>
    stim.setStimSpec('GDcl')
  File "DNstimuli.py", line 45, in setStimSpec
    self.command(1,stimSpec)
  File "DNstimuli.py", line 45, in setStimSpec
    self.command(1,stimSpec)
  File "C:\Program Files\WorldViz\Vizard4\python\vizbdb.py", line 254, in trace_debug
    self.wait_for_input(frame,None)
  File "C:\Program Files\WorldViz\Vizard4\python\vizbdb.py", line 436, in wait_for_input
    id,data = self.wait_for_message(_INPUT_MESSAGES)
  File "C:\Program Files\WorldViz\Vizard4\python\vizbdb.py", line 410, in wait_for_message
    if id in ids:
RuntimeError: testing

The good part here is that "RuntimeError: testing" is what I am throwing from the C++ code (PyErr_SetString(PyExc_RuntimeError, "testing");)

Other things I tried:
- doing data.set( PYTHON_RETURN_OBJECT, PYTHON_INT(-1)); or data.set(PYTHON_RETURN_OBJECT, NULL); after the call to PyErr_SetString did not help (inspired by some reading of how error states should be communicated from Python).
- try/except around the call where the exception is raised. This didn't do anything, as expected when the exception goes unnoticed.

What am I missing? Thanks for your help!

[dcnieho]

Ok, I have a (temporary) solution:

In my C++ code I have defined a helper function modeled after the ones in your viz/python include:

// helper for making the list to communicate an exception
// Returns new reference to list of a python object and a string
inline PyObject* PYTHON_EXCEPTION_STRING_LIST(PyObject* exc, const char * msg)
{
    PyObject *list = PyList_New(2);
    PyList_SET_ITEM(list,0,exc);
    PyList_SET_ITEM(list,1,PyString_FromString(msg));
    return list;
}

I then "raise" my exception by

data.set( PYTHON_RETURN_OBJECT, PYTHON_EXCEPTION_STRING_LIST(PyExc_RuntimeError,"testing");

at the python end, I wrap all my command() calls in a function errorHandler defined as follows:

# function that should be wrapped around any interaction with the stimulus class through command()s
# detect if an error case is communicated, if not, simply pass on the input
def errorHandler(self,input):
    if isinstance(input,list) and issubclass(input[0],BaseException):
        # we have an exception to handle, raise it
        raise input[0](input[1])    # raise exception communicated by C++ code
    else:
        return input

This works also if the C++ code doesn't set any return value as in this case command simply returns None (at least that's the input to errorHandler).

I suppose this is dirty and not how its supposed to be done, but tis a workaround for now. Hope you can show me how to get PyErr_SetString to work!

[farshizzo]

Originally Posted by dcnieho

Is this supposed to be

std::list< osg::ref_ptr< MySensor > > m_sensors;

instead? I don't seem to have a viz::ref_ptr (but do have a viz::Referenced*). In any case, things seem to work fine with osg::ref_ptr

The viz::ref_ptr usage was intentional. You need to #include <viz/ref_ptr> which should come with the SDK.

Originally Posted by dcnieho

Other things I tried:
- doing data.set( PYTHON_RETURN_OBJECT, PYTHON_INT(-1)); or data.set(PYTHON_RETURN_OBJECT, NULL); after the call to PyErr_SetString did not help (inspired by some reading of how error states should be communicated from Python).
- try/except around the call where the exception is raised. This didn't do anything, as expected when the exception goes unnoticed.

What am I missing? Thanks for your help!

You should have to set NULL as the return object. However, I just noticed that Vizard is eating up the exception if NULL is returned by the plugin. We'll need to fix this for the next release. In the meantime I would just raise the exception from the Python wrapper method.

[dcnieho]

Hi Farshizzo,

Thanks! I'll use my workaround for now then. Glad you found why it didn't work!

[dcnieho]

Originally Posted by farshizzo

The viz::ref_ptr usage was intentional. You need to #include <viz/ref_ptr> which should come with the SDK.

You should have to set NULL as the return object. However, I just noticed that Vizard is eating up the exception if NULL is returned by the plugin. We'll need to fix this for the next release. In the meantime I would just raise the exception from the Python wrapper method.

Sorry to dig up an old thread, but i am just testing this again with the newest Vizard 4 and SDK (both freshly downloaded today).

When try to send an exception from my python plugin as follows

PyErr_SetString(PyExc_RuntimeError, "errorstring");
data.set( PYTHON_RETURN_OBJECT, NULL);

I always get a TypeError instead of the exception i'm trying to communicate. The stacktrace always ends like this:

  File "C:\Program Files (x86)\WorldViz\Vizard4\python\viz.py", line 7381, in command
    return _ipcSend(_VIZ_CMDEXTNODE,self.id,command,mesg,x,y,z,w)
TypeError: command must be an int

The next call above that is me calling the C++ code from the extension node, e.g. self.command(1,stimSpec) (my node derives from viz.VizExtensionNode)

I'm not sure what this typeerror is referring to, as my command is correct, it works without error when not trying to return an exception.

Am I doing something wrong?

Best and thanks,
Dee

[farshizzo]

Can you post the full traceback? It looks like you are using passing an invalid parameter to the command method. As the traceback says, the first argument must be an int, and you are most likely passing some other type.

[dcnieho]

Dear Farshizzo,

A full traceback is below. The thing is, i only get this error when i'm trying to throw an exception from my C++ plugin, otherwise all executes as it should. So that shouldn't be me using wrong syntax at the Python side, I think?

** Caught exception in main
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "C:\dat\Vizard\walk_Dee_stimuli\script_walk.py", line 6, in <module>
    main.main()
  File "main.py", line 32, in main
    expt.startSesson()
  File "experiment.py", line 93, in startSesson
    self.stim.setStimSpec(cond.worldSpec)
  File "DNstimuli.py", line 62, in setStimSpec
    self.command(1,stimSpec)
  File "C:\Program Files (x86)\WorldViz\Vizard4\python\viz.py", line 7381, in command
    return _ipcSend(_VIZ_CMDEXTNODE,self.id,command,mesg,x,y,z,w)
TypeError: command must be an int

The self in DNstimuli refers to a class that derives from viz.VizExtensionNode. stimSpec there is a string (or however those are called in Pyhton).

Thanks,
Dee

[farshizzo]

It looks like you are setting the return value incorrectly. Try using the macro for setting return exceptions:

PYTHON_DATA_EXCEPTION(data,PyExc_RuntimeError, "errorstring");

[dcnieho]

Thanks Farshizzo, that did the trick.

Best,
Dee