Seeing yourself as an avatar

[shahargur]

I want to make a user see himself as an avatar in the simulation. It would be great if I can make him see "himself" in the mirror for example.
Any ideas on how I can make it look realistic?
Thanks

[farshizzo]

This is possible using a render to texture technique. I don't have time right now, but I will try to get you an example next week.

[shahargur]

hi,
thanks, please try to send me the example script when you get the time- i'm pretty lost in my efforts. my goal is to run a simulation where the users see themselves as different avatars.
thanks again

[farshizzo]

Here is a sample script that adds a mirror to the gallery environment and adds a self avatar that can be seen in the mirror:import viz
viz.go()

def addMirror(mirror,mat=None):

#If mirror matrix is not specifed, get matrix of mirror object
if mat is None:
mat = mirror.getMatrix()

#Position of mirror
pos = viz.Vector(mat.getPosition())

#Direction mirror is pointing
dir = viz.Vector(mat.getForward())
dir.normalize()

#Quaternion rotation of mirror
quat = mat.getQuat()

#Create render texture
tex = viz.addRenderTexture()

#Create render node for rendering reflection
lens = viz.addRenderNode(size=[1024,1024])
lens.attachTexture(tex)
lens.setInheritView(True,viz.POST_MULT)
lens.disable(viz.CULL_FACE,op=viz.OP_SET_OVERRIDE)

#Setup reflection matrix
rot = viz.Matrix.quat(quat)
invRot = rot.inverse()
lens.setMatrix(viz.Matrix.translate(pos*-1.0)*invRot*viz.Matrix.scale(1,1,-1)*rot*viz.Matrix.translate(pos))

#Setup reflection clip plane
plane = vizmat.Plane(pos=pos,normal=dir)
dist = plane.distance([0,0,0])
lens.clipPlane([-dir[0],-dir[1],-dir[2],dist+0.001])

#Project reflection texture onto mirror
mirror.texture(tex)
mirror.texGen(viz.TEXGEN_PROJECT_EYE)

#Add gallery environment
gallery = viz.add('gallery.ive')

#Use existing painting as mirror and specify the matrix
mirror = gallery.getChild('art04-FACES')
m = viz.Matrix()
m.setPosition(mirror.getPosition(viz.ABS_GLOBAL))
m.setEuler(180,0,0)

#Apply mirror settings to mirror object
addMirror(mirror,m)

#Increase ambient lighting
viz.MainView.getHeadLight().ambient(1,1,1)

#Create a self avatar
avatar = viz.add('vcc_female.cfg')

#Link avatar body to viewpoint
avatarLink = viz.link(viz.MainView,avatar)
avatarLink.setPos([None,0,None]) #Keep avatar on floor
avatarLink.setEuler([None,0,0]) #Only update avatar yaw

[dan12345]

it's beautiful! thanks a lot!!!

[Frank Verberne]

Thanks for posting the code for the mirror! One question though: how can I change the script so that only the head of the avatar will move according to the head movements of the participant instead of the whole body? I tried this:
#Link avatar body to viewpoint
head = avatar.getBone('Bip01 Head')
head.lock()
avatarLink = viz.link(viz.MainView,head)
but this doesn't seem to work. Anybody any suggestions?

[farshizzo]

You can create a timer that manually updates the head bone:def updateHead():
euler = tracker.getEuler()
head.setEuler(euler,viz.ABS_GLOBAL)
vizact.ontimer(0,updateHead)

[Frank Verberne]

Thank you for the code for the timer! Although I had to replace viz.ABS_GLOBAL with viz.ABS_PARENT to make the head stay right on the avatar; when I used viz.ABS_GLOBAL, the head would look upwards instead of the direction of the mainview. Now I still have a problem with the movement of the head: it doesn't move according to the mainview. How can I accomplish that? To make the problem clearer, I've posted my code below.
myIP = socket.gethostbyname(socket.gethostname())

if myIP in RIVERlab1:
print 'I am in RIVERlab 1. I should connect to Intersense tracker.'
#Perform general RIVERlab1 specific things here.
PORT_INTERSENSE = 0
isense = viz.add('intersense.dls')
viz.tracker()
floorY = -1.445233
print '-------'

if myIP in RIVERlab1Clients:
print 'Doing things that only a client in RIVERlab1 should do.'
#Perform RIVERlab1 specific client things here.
print '-------'
else:
print 'Doing things that only a control in RIVERlab1 should do'
#Perform RIVERlab1 specific control things here.
print '-------'

elif myIP in RIVERlab2:
print 'I am in RIVERlab 2. I should connect to DTrack@192.168.2.102'
# Perform general RIVERlab2 specific things here.
tracker = []
vrpn = viz.add('vrpn7.dle')
for trackernr in range(0, 20):
tracker.append(vrpn.addTracker('DTrack@192.168.2.1 02', trackernr))
tracker[trackernr].swapPos([-2,3,1])
tracker[trackernr].swapQuat([2,-3,-1,4]) # Kan het zijn dat niet voor elke sensor een negatieve y en z geldt?
floorY = 0.0
print '-------'

if myIP in RIVERlab2Clients:
print 'Doing things that only a client in RIVERlab 2 should do.'
#Perform RIVERlab2 specific client things here.
viz.link(tracker[0], viz.MainView)
viz.eyeheight(0)
print '-------'
else:
print 'Doing things that only a control in RIVERlab 2 should do.'
#Perform RIVERlab2 specific control things here
viz.link(tracker[0], viz.MainView)
viz.eyeheight(0)
print '-------'

else:
print 'unknown ip'


if myIP in RIVERlab1Clients + RIVERlab2Clients:
print 'Doing things that every client should do.'
#Perform general client things here.
viz.go(viz.HMD | viz.STEREO)
print '-------'

else:
print 'Doing things that every control should do.'
#Perform general control things here.
viz.go(viz.PROMPT)
print '-------'

def addMirror(mirror,mat=None):

#If mirror matrix is not specifed, get matrix of mirror object
if mat is None:
mat = mirror.getMatrix()

#Position of mirror
pos = viz.Vector(mat.getPosition())

#Direction mirror is pointing
dir = viz.Vector(mat.getForward())
dir.normalize()

#Quaternion rotation of mirror
quat = mat.getQuat()

#Create render texture
tex = viz.addRenderTexture()

#Create render node for rendering reflection
lens = viz.addRenderNode(size=[1024,1024]) #set to [1024,1024] in the lab!
lens.attachTexture(tex)
lens.setInheritView(True,viz.POST_MULT)
lens.disable(viz.CULL_FACE,op=viz.OP_SET_OVERRIDE)

#Setup reflection matrix
rot = viz.Matrix.quat(quat)
invRot = rot.inverse()
lens.setMatrix(viz.Matrix.translate(pos*-1.0)*invRot*viz.Matrix.scale(1,1,-1)*rot*viz.Matrix.translate(pos))

#Setup reflection clip plane
plane = vizmat.Plane(pos=pos,normal=dir)
dist = plane.distance([0,0,0])
lens.clipPlane([-dir[0],-dir[1],-dir[2],dist+0.001])

#Project reflection texture onto mirror
mirror.texture(tex)
mirror.texGen(viz.TEXGEN_PROJECT_EYE)

#Add gallery environment
gallery = viz.add('gallery.ive')

#Use existing painting as mirror and specify the matrix
mirror = viz.addTexQuad()
mirror.scale(1.3,3,1)
mirror.setPosition([0, 1, 2])
m = viz.Matrix()
m.setPosition(mirror.getPosition(viz.ABS_GLOBAL))
m.setEuler(180,0,0)

#Apply mirror settings to mirror object
addMirror(mirror,m)

#Increase ambient lighting
viz.MainView.getHeadLight().ambient(3,3,3) #adjust avatar lighting

#Create a self avatar, possible: vcc_male.cfg, vcc_female.cfg, male.cfg, male_bbox.cfg, duck.cfg.
avatar = viz.add('vcc_male.cfg')

def updateHead():
head = avatar.getBone('Bip01 Head')
head.lock()
euler = tracker[0].getEuler()
head.setEuler(euler,viz.ABS_PARENT)

vizact.ontimer(0,updateHead)
For the record: I'm testing in RIVERlab2 and I'm only using one tracker (tracker[0]). The mainview is linked to tracker[0] with viz.link(tracker[0], viz.MainView) and the head should be updated according to the coordinates of tracker[0]. However, when turning the tracker around it's pitch, the yaw of the head is changed, which should also be it's pitch. I hope the problem is clear; if not, I will try to clarify it.

[farshizzo]

I made a mistake in my last post, it should be viz.AVATAR_WORLD, not viz.ABS_GLOBAL

[Frank Verberne]

Changing viz.ABS_GLOBAL into viz.AVATAR_WORLD worked perfectly, thanks! Together with a collegeau I also managed to get the same result using the viz.link() function:
for i in range(0, len(link_bones)):
body = avatar.getBone(link_bones[i])
Link = viz.link(tracker[i],body)
Although not all bones respond properly to its tracker yet, we are going to work on that tomorrow. If I run into more problems, I will post them here. Thank you for helping me thus far;)!

[Frank Verberne]

Today it appeared that the function of my last post doesn't work after all. I still had the old function using the timer active while testing the link command.