WorldViz User Forum

WorldViz User Forum (https://forum.worldviz.com/index.php)
-   Plug-in development (https://forum.worldviz.com/forumdisplay.php?f=8)
-   -   Plugin For Headtracking with webcam via FaceAPI (https://forum.worldviz.com/showthread.php?t=2656)

Adam 03-30-2010 03:13 PM

Plugin For Headtracking with webcam via FaceAPI
 
1 Attachment(s)
I would like to be able to be able to read the head position (x,y,z) and rotation values (y,p,r) from a webcam using FaceAPI and apply that data to my viewpoint in vizard and I am at a loss as to how to do this.

This video illustrates what I would like to be able to do. http://www.youtube.com/watch?v=uA8ZXPS6onk

I have found some google code projects that provide this position and orientation data via a socket but I have never worked with sockets before.

http://code.google.com/p/6dofstreamer/

I've attached an image of what the socket outputs.

How would I go about either accessing this data from the socket to update the viewpoint, or develop my own faceapi plugin to do this automatically.

Many thanks in advance to those more knowledgeable than myself.

farshizzo 03-31-2010 05:54 PM

Do you have more information about the FaceAPI socket protocol? Either way, you should be able to use the built-in Python socket module to receive data over the network.

JvdBosch 04-02-2010 01:47 AM

Adam, I've sent you a private message.

Adam 04-16-2010 11:55 AM

I am trying to work that FACEAPI integration for headtracking via streaming position and orientation data from a socket and can't quite get it to work.

Could you look over my code and let me know what I may be doing wrong? The socket works and I can access the data, but I can't seem to get it to apply to my viewpoint.

Thanks,

-Adam

Code:

# Import Modules
import viz
from socket import *
import string
import math

# Set Global Variables
global host
global port 
global buf
global addr
global HeadPos
global HeadRot
global pos
global euler
global view

# Run Vizard in Fullscreen
viz.go()
view = viz.MainView
viz.mouse(viz.OFF)
viz.mouse.setVisible(False)

# Load vrml and locate objects
viz.add('tut_ground.wrl')
pole = viz.add('pole.wrl')
pole.setPosition(0,0,4)



# Set the socket parameters
host = "localhost"
port = 29129
buf = 1024
addr = (host,port)

# Create socket and bind to address
UDPSock = socket(AF_INET,SOCK_DGRAM)
UDPSock.bind(addr)

# Receive messages
while 1:
        data,addr = UDPSock.recvfrom(buf)
        if not data:
                print "Client has exited!"
                break
        else:
                #print "\nReceived message '", data,"'"
                cp = str(data).split(' ')
                headposx = cp[0]
                headposx = float(headposx)
                headposy = cp[1]
                headposy = float(headposy)
                headposz = cp[2]
                headposz = float(headposz)
                headrotx = cp[3]
                headrotx = float(headrotx)
                headroty = cp[4]
                headroty = float(headroty)
                headrotz = cp[5]
                headrotz = float(headrotz)
                HeadPos = [headposx , headposy , headposz]
                print "\nHeadPos = ", HeadPos,"'"
                HeadRot = [headrotx , headroty , headrotz]
                print "\nHeadRot = ", HeadRot,"'"
                updateview()
       
# Close socket
UDPSock.close()

def updateview():
        viz.reset(viz.HEAD_ORI|viz.BODY_ORI|viz.HEAD_POS)
       
        #update orientation
        euler = view.getEuler(viz.Head_ORI)
        euler[0] += HeadRot[0]
        euler[1] += HeadRot[1]
        euler[2] += HeadRot[2]
        print "\nEuler = ", euler
        view.setEuler(euler,viz.Head_ORI)
       
        #update position
        pos = view.getPosition(viz.Head_POS)
        pos[0] += HeadPos[0]
        pos[1] += HeadPos[1]
        pos[2] += HeadPos[2]
        print "\nPosition = ", pos
        view.setPosition(pos[0],pos[1],pos[2])

vizact.ontimer(0,updateview)



All times are GMT -7. The time now is 08:23 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Copyright 2002-2023 WorldViz LLC