record and playback head-tracked data

[michaelrepucci]

Hi Vizards, I'm a little stuck on a project I've been trying to implement. I have a Power Wall setup with head-tracking in which I'd like to record the head position and orientation while a user walks around a virtual environment. Then, while the user remains stationary, I'd like to play back that same scene (with frustum and view updates) to the user.

I can setup my cave, link the head-tracker to it, and record the head-tracking data just fine with a onFrameUpdate callback. But then when I try to remove the head-tracker (programmatically), and use the recorded coordinates to update the cave frustums (cave.update) and Vizard MainView (viz.MainView.setPosition) it doesn't behave as I expect, and the scene simply doesn't update.

Maybe I'm not understanding what the right approach to this problem is at all. Perhaps I should be using some sort of camera? Could any of you Vizard guru's please give me some help? Thank you!

[farshizzo]

It's hard to tell what you are doing wrong without seeing any sample code. I've attached a sample script that will record the tracker data and play it back. Press space key to start/stop recording. Once the recording has stopped it will automatically playback the tracking data. Hopefully this will give you some ideas on how to implement what you are trying to accomplish.

import viz
import vizcave
import viztracker
import viztask

#Width and height of power wall
WIDTH = 2.0
HEIGHT = 1.0
DISTANCE = 1.0

#Initialize graphics window
viz.go()

#Add environment model
viz.add('gallery.ive')

#Create tracker object using KeyboardPos tracker
tracker = viztracker.KeyboardPos()

#Create single power wall
PowerWall = vizcave.Wall(   upperLeft=(-WIDTH,HEIGHT,DISTANCE),
                            upperRight=(WIDTH,HEIGHT,DISTANCE),
                            lowerLeft=(-WIDTH,-HEIGHT,DISTANCE),
                            lowerRight=(WIDTH,-HEIGHT,DISTANCE),
                            name='Power Wall' )

#Create cave object with power wall
cave = vizcave.Cave()
cave.addWall(PowerWall)

#Use tracker for automatically updating cave
cave.setTracker(pos=tracker)

#Create vizcave.CaveView object for manipulating viewpoint in cave environment
origin = vizcave.CaveView(tracker)

#Translate view origin 1 meter back
origin.setPosition(0,0,-1)

def SetCaveTracker(posTracker):
	cave.setTracker(pos=posTracker)
	origin.setTracker(posTracker)

def RecordPlayTask():
	
	#Add dummy node for using as playback tracker
	playbackTracker = viz.addGroup()

	#Text for displaying record/playback status
	statusText = viz.addText('',parent=viz.ORTHO,color=viz.RED,fontSize=48)
	
	while True:
		
		#Wait for spacebar to start recording
		yield viztask.waitKeyDown(' ')
		
		statusText.message('Recording')
		
		#Start recording callback
		recordedData = []
		def _RecordTrackingData():
			recordedData.append( (tracker.getPosition(), tracker.getQuat()) )
		recordCallback = vizact.onupdate(0,_RecordTrackingData)
		
		#Wait for spacebar to stop recording
		yield viztask.waitKeyDown(' ')
		
		#Stop recording callback
		recordCallback.remove()
		
		statusText.message('Playing')
		
		#Set playback tracker
		SetCaveTracker(playbackTracker)
		
		#Playback recorded data
		for pos,ori in recordedData:
			playbackTracker.setPosition(pos)
			yield None
	
		statusText.message('')
		
		#Restore original tracker
		SetCaveTracker(tracker)
		
viztask.schedule( RecordPlayTask() )

[michaelrepucci]

Okay. Interesting. I was able to get a stripped-down example working, though my approach was slightly different than yours. So for the forum record I will post it here, then I just have a few questions about the differences. FYI, the sunyobjects module is just full of on-the-fly objects whose appearance should be obvious from the class name.

from __future__ import division #implements "/" as division without truncation regardless of its operands
import viz #basic Vizard module
import vizact #Vizard actions module
import viztask #Vizards task execution module
import vizcave #Vizard cave module
import vizinfo #Vizard 2D GUI commands
import viztracker #Vizard position/orientation tracker module
import sunyobjects

def onFrameUpdate(event):
	global headPosition, headOrientation
	
	#save head position and orientation
	headPosition += [tracker.getPosition()]
	headOrientation += [tracker.getQuat()]

def run():
	for i in range(2):
		if i%2:
			yield viz.message('Playback Head Position')
			for hp, ho in zip(headPosition,headOrientation):
				cave.update(hp,ho)
				viz.MainView.setPosition(hp)
				viz.MainView.setQuat(ho)
				yield viztask.waitDraw()
		else:
			yield viz.message('Recording Head Position (10 seconds)')
			cave.setTracker(pos=tracker,ori=tracker) #update frustums based on head-position
			view = vizcave.CaveView(tracker) #update view based on head-position
			viz.callback(viz.UPDATE_EVENT,onFrameUpdate)
			yield viztask.waitTime(10)
			viz.callback(viz.UPDATE_EVENT,None)
			cave.setTracker(pos=None,ori=None) #prevent update frustums based on head-position
			view.remove() #prevent update view based on head-position

#global variables
headPosition = [] #head position on each frame
headOrientation = [] #head orientation on each frame

#display keyboard mapping
keyMap = vizinfo.add('Left/Right = Q/E\n'+
	'Up/Down = R/F\n'+
	'Forwards/Backwards = W/S\n\n'+
	'Roll Left/Right = G/J\n'+
	'Pitch Up/Down = Y/H\n'+
	'Yaw Left/Right = A/D')
keyMap.title('Keyboard Tracker Mapping')
keyMap.shrink()

viz.go()
viz.eyeheight(1)
viz.MainWindow.mouse(viz.OFF)
viz.mouse.setVisible(viz.OFF)

#keyboard simulated tracker
tracker = viztracker.add()

#setup cave with wall and tracker (optional)
wall = vizcave.Wall('wall',[-1,1,0],[1,1,0],[-1,-1,0],[1,-1,0])
cave = vizcave.Cave()
cave.addWall(wall)

cube = sunyobjects.CompositeCube(0.5,0.01)

viztask.schedule(run()) #since control passes immediately do not place critical code after this point

So one fundamental difference is that during playback you use a node3d object to link to the cave (cave.setTracker and vizcave.CaveView), and then change that node's position, whereas I use cave.update and viz.MainView.setPosition directly. Aside from personal preference is there any performance reason to do this one way or the other?

A more minor difference is that during playback you yield on None after each position update, whereas I yield on viztask.waitDraw(). What is the difference between these approaches, and which would ensure that each frame of the playback was identical to the recorded frames?

Finally, and this perhaps is not a difference at all, you use vizact.onupdate, while I use viz.callback(viz.UPDATE_EVENT). Is there a reason to prefer one command over the other, or are they completely identical?

Thanks for all the help!

[farshizzo]

So one fundamental difference is that during playback you use a node3d object to link to the cave (cave.setTracker and vizcave.CaveView), and then change that node's position, whereas I use cave.update and viz.MainView.setPosition directly. Aside from personal preference is there any performance reason to do this one way or the other?

There is not major performance reason. The biggest reason is to reduce the amount of code. Your method uses two different code paths to update the cave, depending on whether you are recording or playing data.

A more minor difference is that during playback you yield on None after each position update, whereas I yield on viztask.waitDraw(). What is the difference between these approaches, and which would ensure that each frame of the playback was identical to the recorded frames?

I would not use viztask.waitDraw unless you are interested in the time that the frame is drawn to screen. If you don't care about the time, then I highly recommend using viztask.waitFrame(), or just yield None.

Finally, and this perhaps is not a difference at all, you use vizact.onupdate, while I use viz.callback(viz.UPDATE_EVENT). Is there a reason to prefer one command over the other, or are they completely identical?

I generally recommend using the vizact module instead of using viz.callback. Using viz.callback might break your code in certain cases, because only one function can be registered at a time with viz.callback. So if you end up using it in multiple parts of your script you might end up inadvertently unregistering an update callback.