Oculus Rift DK2 + gamepad controller

[rdkirkden]

Hi,

I’m trying to build an environment in which the subject wears an Oculus Rift DK2 headset and is additionally able to navigate using a gamepad controller (Logitech Dual Action). The controller should allow the subject to move along both horizontal axes using the left handle and also turn (i.e. rotate about the vertical axis) using the right handle.

The problem I have encountered is that when the subject turns using the controller, the horizontal axes are not re-aligned to the subject’s new orientation, with the consequence that after a 90 degree turn, the left/right handle action that previously caused movement along the x-axis now causes movement along the y-axis.

In an attempt to narrow down the cause of the problem, I started with the ‘navigation.py’ example script (http://docs.worldviz.com/vizard/#Nav...ght=navigation), which gives complete gamepad control over a view on the monitor, but does not incorporate a headset. To this, I added code from the ‘oculusExample.py’ script (Program Files\WorldViz\Vizard5.2\examples\devices\, cited in http://docs.worldviz.com/vizard/#ocu...=oculusExample), which incorporates an Oculus Rift DK2 headset, but has only limited keyboard controls (to move along the horizontal axes, but no turning). The problem emerged as soon as I instructed the headset sensor to link with the ‘MainView’. I attach two code listings: in the first, there is still full gamepad control but no headset control over the view because the link has not yet been made; in the second, the headset has been linked in and is working properly, but the problem with gamepad control has emerged.

You can see that in addition to adding the headset instructions, I’ve also made a few changes to the gamepad control code in the ‘UpdateJoystick’ function. This was necessary to get as much functionality as I could from the gamepad and headset, leaving just the problem with turning that I could not solve. The changes that I succeeded in making were as follows.

1) The ‘viz.MainView.move’ statement didn’t work at all after linking the Oculus sensor, so the gamepad could not be used to move along the horizontal axes. This was solved by using a ‘navigationNode.setPosition’ statement instead. I’ve noticed that the ‘oculusExample.py’ script similarly uses ‘navigationNode.setPosition’ to get its keyboard controls working, although I don’t understand why this is necessary. (In contrast, the ‘viz.MainView.setEuler’ statement worked fine, so the gamepad could be used to rotate the view. Note that I’m using the ‘z’ variable here, not the ‘twist’ variable, because the Device Calibration Wizard got the horizontal and vertical actions of the right handle mixed up.)

2) After changing this, the gamepad could be used to move along the horizontal axes, but when the subject turned their head the horizontal axes were not re-aligned to the subject’s new orientation. To solve this, I had to incorporate the subject’s current yaw value into the ‘navigationNode.setPosition’ statement, using code that I do not understand, but copied from the keyboard controls in the ‘oculusExample.py’ script.

To clarify my present situation, the horizontal axes are now being re-aligned when the headset turns, but not when the view is rotated using the right gamepad handle.

I’ve tried everything I can think of to get his working, but with no success. A large part of the problem is that I don’t understand all of the code, because the documentation in the Vizard 5 Reference is not very detailed.

I should add that I’m using Vizard 5.2 and Oculus SDK 0.5. Before attempting to write code and test any specific functionality, I was advised on this forum to update to the latest versions of both Vizard and Oculus; but I’m reluctant to do this unless I’m certain it is necessary, just in case it causes problems for some other hardware that I may need to use (some old NVIDIA 3D VISION 2 glasses). The two example scripts mentioned above each works perfectly on my system, it’s just that I can’t quite get them to fit together.

Any advice appreciated!

Richard.

[rdkirkden]

Well, I found a solution in a related thread: http://forum.worldviz.com/showthread.php?t=5662. This involved using the Vizconnect utility to configure the behaviour of my hardware (HMD, tracker and gamepad) for me. Even this was not straightforward, as the documentation in Vizard is so poor. I found the easiest way was to use the wizard (option 2 on the startup page) to set up a basic configuration and then modify a few settings in the advanced configuration menu (option 3). In particular, I had to decrease the ‘rotation acceleration’, ‘max rotation speed’ and ‘rotation auto breaking drag coeff’ in the ‘Transports’ settings to prevent a feeling of nausea when using the gamepad to rotate the view.

Vizconnect proved to be great for getting my hardware devices to interact correctly with one another, but I found that it presented some difficulties when it came to accessing and controlling objects in the script. This is because it created a number of objects (handles and links and so on for my devices and for an avatar) which it was not obvious how to reference; and also because some basic methods of ‘viz’ and ‘viz.MainWindow’ would no longer work.

In case anybody else is following this thread and has not used Vizconnect before, I’ll mention a couple of issues that I managed to solve either by stumbling across nuggets buried in the Vizard documentation or elsewhere, or by examining the Vizconnect configuration file script, or by trial and error.

1) To get rid of the disembodied gloved hand that vizconnect insists on displaying: ‘vizconnect.getAvatar().getNode3d().visible( viz.OFF )’. (Warning: do not delete the glove because there’s a hideous zombie hand underneath!!)

2) To set the avatar’s initial viewpoint in the environment (normally achieved using ‘room = viz.addChild(‘name’)’ and ‘room.setPosition([x,y,z])’), you have to first reference the tracker’s link object (‘tracker_link = vizconnect.getTracker('head_tracker').getLink()’), then straighten out the view because vizconnect unaccountably makes it crooked (‘tracker_link.reset(viz.RESET_ORI_HEADING)’), and finally set the viewpoint you want using ‘postTrans’ (e.g. tracker_link.postTrans([0,1.8,-8])’).

3) Collision control can’t be set up using the normal ‘viz.collision(viz.ON)’ command because vizconnect prevents this from working. It is also impossible to separately set the collision status of objects that are later added to the environment, although their gravity status can be set. Instead, it is necessary to set a uniform control collision for the whole environment by importing the ‘view_collision’ script that is in Vizconnect’s ‘utils’ folder. Instructions for using this script are at: https://www.worldviz.com/february-te...int-collision/. But note that the version of ‘view_collision’ that comes with Vizard 5.2 doesn’t work in several respects. I ended up using the version that came with Vizard 5.5 instead, and this was fine in most respects, except that it causes the avatar to bounce off some objects in the environment (though not the walls) with quite a judder. This last issue is the only thing I’ve not been able resolve (I haven’t needed to yet as in the environment I’m working on the objects are all low enough that the avatar can float over them – it’s the walls that I wanted to make collision-proof).

[rdkirkden]

I was unhappy that a slight feeling of nausea remained when using the gamepad to rotate the view, even after optimising the transport’s rotation settings in Vizconnect, so I tried updating my version of Vizard to 5.7 and my Oculus Rift SDK to 1.16. This made absolutely no difference. However, it did affect a couple of other things…

1) It greatly decreased the degree to which the avatar bounced off objects in the environment when colliding with them. This was only true when using the Vizard 5.5 version of the ‘view_collision’ script, not when using the Vizard 5.7 version. As with most things in Vizard, I have no understanding of this, but at any rate it was fortuitous.

2) The view that appeared on the desktop monitor became a stereo view, instead of the mono view that I had before. This was an unwanted change. I managed to get the mono view back by adapting some code that I found in another thread (http://forum.worldviz.com/showthread...ght=vizconnect). I added the following lines of code to my main Vizard program:

hmd = vizconnect.getRawDisplayDict()['main_display'].displayNode
hmd.setMonoMirror(True)

As regards the persistent problem of slight nausea when using the gamepad to rotate the view, I’ve found that this is particular to a certain kind of rotation. Nausea occurs when the game controller causes the avatar to turn on the spot (the sort of rotation you get with the WandMagicCarpet and Driving transports), but not when the controller causes the avatar to orbit around some other location in the environment (which happens with the Orientation and Orbit transports). The feeling of nausea seems to be associated with the way in which the view whirls past whilst the avatar is being turned (this can be reduced by slowing rotation speed and acceleration), and also with a bounce that occurs when the turning stops (the bounce can be changed to a gentle deceleration by decreasing rotation drag). I had exactly the same experience when I tried out the ‘VizConnect -> Inputs and Transports’ tutorial that has the avatar travelling inside a car, so it’s not something odd about my program. I should stress that there is absolutely no sensation of nausea when I turn my head, only when my view is being rotated by the game controller. I wonder if some degree of nausea is unavoidable for this kind of involuntary view rotation. Perhaps it’s normal to feel nauseous when the environment is whirling past, but not when performing a natural head turn, or orbiting a fixed point that one’s eyes are drawn to. I have no previous experience of VR environments, so this is new to me.

My ‘optimised’ settings for the WandMagicCarpet transport in Vizconnect were: ‘rotation acceleration’ = 30; ‘max rotation speed’ = 32; ‘rotation auto breaking drag coef’ = 0.05. I left the ‘rotation drag coef’ setting at its default value of 0.0001. Turning is admittedly quite slow.

As a side issue, I found a better way to hide the disembodied gloved hand:

right_hand = avatar.getAttachmentPoint('r_hand').getNode3d()
right_hand.remove()

[Jeff]

When the viewpoint in the virtual world moves out of sync with the user's physical movements nausea may occur and the effect can vary from person to person. It's best to limit or slow down movements that are virtual only. If you need the user to explore a large environment, one workaround maybe to directly place the user at different locations. This is possible using vizconnect viewpoints. Then have the user explore a smaller region around each location with physical tracking only.

[rdkirkden]

Hi Jeff,

Thanks for clarifying that some degree of nausea may be unavoidable when using a gamepad controller to rotate the view. The environment the user is exploring is quite small, and the feeling of nausea is very mild (at least for me) when the rate of turning is slowed down a lot; but slowing the turning down is itself undesirable for the experiment and I want to allow faster turning if possible. To this end, I've cleared an area around the user's chair and dropped the HMD cable from the ceiling so as to allow the user to rotate his/herself instead of using the gamepad to rotate the view. However, I've found that when facing directly away from the Oculus tracker a glitch occurs: when examining a nearby object, moving the head forwards to look at it more closely causes the object to retreat, while moving the head backward causes it to advance. The glitch can be rectified by using the gamepad to rotate the view whilst turning the head to keep looking at the object: at some point the view will 'bump' and the glitch will be fixed. But the glitch comes back if the user faces away from the tracker again. A combination of turning the head, so that the user is no longer facing directly away from the tracker, and using the gamepad to rotate the view seems to be required, since merely turning the head to face the tracker fails to rectify the glitch. I'm not sure what's going on here. I'm displaying the camera bounds using <hmd>.addCameraBounds() and it doesn't look as if I'm straying outside the bounds. Is the tracker just unable to cope when the user is facing directly away, or is there some software routine I can use to automatically correct this glitch when it occurs?

Thanks,
Richard.