Can I get the Walking tracker’s turning motion while using an Oculus VR DK2 tracker?

[rdkirkden]

Long title: Can I achieve the same turning motion that is implemented by the Mouse and Keyboard Walking tracker while also using an Oculus VR DK2 tracker?

In Vizard 5, I am using the ‘Oculus VR DK2’ tracker in combination with a ‘WandMagicCarpet’ transport that receives joystick input (set up using Vizconnect) so that the avatar can move backwards/forwards/left/right and turn with the joystick in addition to looking around with the Oculus HMD. However, the turning motion can cause motion sickness, and I think this might be associated with the accelerations and decelerations that the transport implements. I have noticed that the ‘Mouse and Keyboard Walking’ tracker produces a different sort of turning motion with the mouse that feels more like looking around than turning the body and seems to produce less motion sickness. I would like to implement this sort of turning in combination with the ‘Oculus VR DK2’ tracker, but I can’t find a way to do this in Vizconnect. (1) I was able to make the ‘Mouse and Keyboard Walking’ tracker work alongside the existing ‘Oculus VR DK2’ tracker by combining them in the Scene Graph, but no matter how I arranged them they interacted in nauseating ways. I need them to work independently. (2) It would seem to make more sense to have ‘walking’ as a transport than as a tracker, as transports and trackers can be combined. There is a ‘Walking’ transport available in Vizconnect, but this gives the same kind of turning as the ‘WandMagicCarpet’ transport. (3) Since the ‘Mouse and Keyboard Walking’ tracker uses a mouse input for turning, I thought of adding a mouse input, but there is no such Input option available in Vizconnect. The nearest is ‘Mouse Button’, and this gives the same sort of turning as the joystick when the left mouse button is assigned to ‘turnLeft’ and the right button to ‘turnRight’ in the transport. Overall, I think I need either: (1) a way to have two trackers operating independently; or (2) a transport with the same physics as the ‘Mouse and Keyboard Walking’ tracker. Please can you tell me if this is possible?

[rdkirkden]

I thought I’d add a bit of further information to clarify the sort of turning movement that I’m after. I’d be interested to know if anybody else has an opinion on this, even if there isn’t a simple solution.

The transports seem to implement a particular type of turning, which is based on the physics of acceleration and deceleration. The game controller’s joystick position does not directly control velocity. Rather, it controls acceleration, which in turn affects velocity. In contrast, with the ‘Mouse and Keyboard Walking’ tracker, the turning behaviour seems intended to simulate looking around rather than turning the body, and it does not involve gradual acceleration and deceleration. It is normally controlled with a mouse, not a joystick, and I’m not sure if it would work with a joystick. But mouse movements seem to translate more or less directly into changes in viewing angle, rather than simulating the physics of turning. It’s like some computer games that use the mouse to look around and the keyboard to move relative to the direction faced. The viewing direction changes more or less immediately and the turning process is as fast as the mouse movement. I’m inclined to think that this is more naturalistic, as it like turning your head: you more or less switch from one view to another while barely registering the intermediate views. It seems to me that it produces less nausea.

Googling VR sickness, I’ve subsequently come across the ‘Oculus Best Practices’ document, which indicates that VR sickness has many causes, but highlights acceleration as one of them. It goes on to say: “An instantaneous burst of acceleration is more comfortable than an extended, gradual acceleration to the same movement velocity.” This seems to bear out my experience. The transports simulate the physics of motion for maximal realism, but this has a cost. The document mentions a novel approach aimed at mitigating VR sickness: “Rather than smoothly rotating, pressing left or right on a controller causes the camera to immediately jump by a fixed angle (e.g., 30°) in the desired direction.” This is a more extreme (i.e. less realistic) solution than I had in mind, but the general thrust of this approach (i.e. achieving more or less instantaneous turning) is similar.

If there is not an easy solution (i.e. a way to combine trackers, or a ready-made transport), then I suppose it would be necessary to rewrite some of the code that controls how one of the existing transports works.

[sado_rabaudi]

Hi,

You can use a "merged" tracker that will combine the orientation from one tracker and the position from another. For instance you could use the "Virtual Mouse Orientation" tracker to use the mouse orientation and then use the position from the DK2.

Best,
Sado

[rdkirkden]

Hi Sado,

Thanks so much for your reply.

The ‘Virtual Mouse Orientation’ tracker gives the same kind of rotation as the ‘Mouse and Keyboard Walking’ tracker, which is great. But I’m struggling to combine this with the ‘Oculus VR DK2’ tracker, as the trackers seem to affect one another. Testing in the Vizconnect interface, the best arrangement I’ve found is as shown in the image.

In this arrangement, the ‘Virtual Mouse Orientation’ tracker behaves well, and the ‘Oculus VR DK2’ tracker seems fine except that turning the head from side to side causes roll in addition to yaw. Is there a better arrangement that I could use? Or perhaps I could resolve this by preventing any rolling motion from occurring by adding some code somewhere in the Vizconnect file or my main Vizard program?

Richard

[sado_rabaudi]

One way to do this is to first create a group tracker in vizconnect that will contain the modified tracking data without roll. Next, in the avatar animator set the avatar head to move with this group tracker. Then in a callback function, which could be in the postInit or the application script, set the group tracker's position,yaw, and pitch using data coming from the Oculus tracker. Here's an example function:

def postInit():
"""Add any code here which should be called after all of the initialization of this configuration is complete.
Returned values can be obtained by calling getPostInitResult for this file's vizconnect.Configuration instance."""

Group tracker need to be added to vizconnect and used in animator
groupTracker = vizconnect.getRawTracker('headTrackerGroup')
steamHeadTracker = vizconnect.getRawTracker('head_tracker')

def onUpdate():
pos = oculusHeadTracker.getPosition(viz.ABS_GLOBAL)
ori = oculusHeadTracker.getEuler(viz.ABS_GLOBAL)

Set roll to 0
ori = [ori[0], ori[1], 0]
groupTracker.setPosition(pos, viz.ABS_GLOBAL)
groupTracker.setEuler(ori, viz.ABS_GLOBAL)

vizact.onupdate(vizconnect.PRIORITY_ANIMATOR+1, onUpdate)

return None

[rdkirkden]

Hi Sado,

Thanks for getting back to me with such detailed advice! I’ve got it working now. I did one thing differently by mistake, but then left it as it was because I think it suits the design of my program. I’ll outline what I did, in case this is useful for anybody else. I admit that the things I did that deviated from your advice were pretty haphazard, but it worked in the end.

The main thing I did differently was that I used the group tracker to prevent roll on the mouse controls instead of on the Oculus HMD. I realise this makes no sense, given that my objective was to prevent rolling when the HMD turned from side to side – as I said, it was a mistake! But in my program I want to restrict the mouse controls to yaw only, and when I used the group tracker to prevent both roll and pitch movements on the mouse, this somehow stopped the HMD from producing the unwanted roll when turning from side to side. I should add that I had no choice but to prevent pitch movements on the mouse, because they were being accompanied by an unpleasant lurching forward/rolling behaviour.

I intend to keep the transport for linear movements, i.e. backwards, forwards, left and right. I’m going to upgrade from the DK2 to a Rift S and when I do I hope to replace my gamepad controller with a touch controller, so that the player can have the touch controller in one hand and the mouse in the other. Hopefully it is possible to use just one touch controller – I haven’t tried this yet. The keyboard would be an alternative if I used the Mouse and Keyboard Walking tracker instead of the Mouse Orientation tracker, but I think a touch controller thumbstick would be easier to use while wearing an HMD. Also, I want to use one of the keys on the controller.

I set up the vizconnect scene graph as shown in the image. I set up the vizconnect avatar animator with the head tracker, as setting it up with the mouse orientation group tracker caused some unpleasant effects. The head tracker was specified in the transport settings. Then I added the following code to the postInit() function of the vizconnect file:

def postInit():
	"""Add any code here which should be called after all of the initialization of this configuration is complete.
	Returned values can be obtained by calling getPostInitResult for this file's vizconnect.Configuration instance."""
	import vizact
	groupTracker = vizconnect.getRawTracker('mouse_orientation_group')
	mouseOrientationTracker = vizconnect.getRawTracker('mouse_orientation')
	def onUpdate():
		pos = mouseOrientationTracker.getPosition(viz.ABS_GLOBAL)
		ori = mouseOrientationTracker.getEuler(viz.ABS_GLOBAL)
		#Set pitch and roll to 0
		ori = [ori[0], 0, 0]
		groupTracker.setPosition(pos, viz.ABS_GLOBAL)
		groupTracker.setEuler(ori, viz.ABS_GLOBAL)
		return None
	vizact.onupdate(vizconnect.PRIORITY_ANIMATOR+1, onUpdate)
	return None

[rdkirkden]

Having revisited my program after a long delay, I’ve realised there is one problem with how it behaves. As long as the avatar remains in the starting position, rotation with the mouse works fine. But if the player uses the game controller to move the avatar backwards/forwards/sideways and then uses the mouse to rotate, instead of rotating on the spot, the avatar turns in a circle whose centre is the starting position. It is as if the mouse orientation tracker does not know that the avatar’s position has changed. I have tried setting the group tracker’s position equal to the head tracker’s position, instead of the mouse orientation tracker’s position, but this made no difference. It also occurred to me that I might need to get the position of the wrapped head tracker, as opposed to the raw head tracker, since position changes don’t always affect a raw tracker’s position values, but this did not help either. I attach my Vizconnect file in .txt format. Please can anybody give me advice?

[rdkirkden]

By trial and error, I managed to create a Vizconnect file that:

1) uses a mouse_orientation tracker for turning;
2) also uses an Oculus DK2 tracker for turning (both head and body, not just the head*);
3) turns on the spot, not in a circle.

It was necessary to use a transport (game controller thumbstick) for translational movement, as the keyboard component of a mouse_and_keyboard_walking tracker did not integrate well with the Oculus DK2 tracker. Although translational movement with the transport is associated with undesirable accelerations, it is possible to tweak the settings to minimise the nauseating effects of forwards/backwards acceleration (e.g. auto breaking drag coef = 1, acceleration = 6, max speed = 4). It is only the use of a transport for turning that really needs to be avoided, as the nauseating effects of acceleration when turning cannot be minimised by tweaking settings.

*If the HMD is only required to turn the head, this is much simpler to achieve, using just a ‘mouse_and_keyboard_walking’ tracker and no transport. I suppose this is a naturalistic set-up, where the person can turn their head to look around as they continue to walk in the direction the body is facing. But for a seated participant wearing a HMD, it can be quite difficult to precisely re-orient the head in a forward-facing direction after turning it, so that the participant often ends up moving obliquely to the direction they are facing, and this can become nauseating. Therefore, I required the HMD to turn the body in the same way as the mouse, so that the avatar always walks in the direction they are facing. This can only be achieved via a transport.

To achieve turning on the spot, not in a circle, it was necessary to make the ‘mouse_orientation’ tracker (more precisely, the group tracker that took the yaw value of the ‘mouse_orientation’ tracker) the child of the transport by switching the tracker and the transport around on the inheritance tree (see attached image). This resolved the turning-in-a-circle problem, because the mouse could now see the translational movements generated by the transport. However, walking direction (with the transport) now ignored turning with the mouse. So it was necessary to add some code to the ‘postInit’ method to set the transport’s orientation equal to the ‘mouse_orientation’ tracker’s current orientation.

Moving the mouse now changed walking direction, but for some reason that I don’t understand a mouse rotation of x degrees resulted in a walking direction rotation of x/2 degrees. It was as if the mouse turned the head at twice the rate of the body. In order to fix this, I had to use some more code to recalculate the direction in which the body was facing according to the orientations of the Oculus DK2 and mouse. This was a pretty crude hack, but I couldn’t see another way.

If body yaw <= 180:
body yaw = mouse yaw + head yaw 

If body yaw > 180:
body yaw = mouse yaw + head yaw – 360

(As the view is rotated, instead of increasing from 0 to 360 degrees, the yaw increases from 0 to 180 degrees, then changes abruptly from 180 to -180 and continues to increase from -180 to 0.)

The resulting Vizconnect program behaves well in the Vizconnect Interface. Unfortunately, it does not behave so well when I try to integrate it with my main Vizard program. It’s fine as long as my main Vizard program does not attempt to set/reset the avatar’s viewpoint. I have not managed to resolve this yet and frankly it’s looking very difficult. Anyway, I attach my Vizconnect program in case anybody finds it useful.