WorldViz User Forum  

Go Back   WorldViz User Forum > Vizard

Reply
 
Thread Tools Rating: Thread Rating: 2 votes, 5.00 average. Display Modes
  #1  
Old 09-16-2005, 09:37 PM
vsully vsully is offline
Member
 
Join Date: Sep 2004
Posts: 24
Send a message via AIM to vsully
delays with <avatar>.act and speech

I'm using <avatar>.speak() and <avatar>.act() to get an avatar talking. My problem is that with the sound files I have, (2-3 minute, 2-4 MB), the mouth movements will often end up with a ~5 sec lag by the end of the audio file.

Near 100% sync would be nice if possible, the other solution might involve clearing the face morphs when one audio file ends in order to begin and re-sync the next audio file, without the ~5 sec break in speech while the mouth wags in silence...

Towards the second solution, I'm using <avatar>.clear() with viz.ANIMATION and viz.CURRENT_ANIMATION, but it doesn't seem to be catching the mouth morphing.

Thanks
Reply With Quote
  #2  
Old 09-19-2005, 03:16 PM
farshizzo farshizzo is offline
WorldViz Team Member
 
Join Date: Mar 2003
Posts: 2,849
Hi,

Are you using the built-in head of the avatar or are you adding a vzf head? To clear the morph of a vzf head, you need to call the clear command on the face object of the avatar. If you are using the built-in head then you need to get the jaw bone of the avatar and reset its rotation to the following axis angle rotation [1,0,0,97.35].

However, it should be possible to keep the mouth movement synced. The code for the speak action is located in vizact.py. It currently uses the viz.playsound function to start the sound and doesn't do any resyncing afterwards. Instead of doing this you could add an audio object and periodically check what its current position is and resync the mouth movement with it. When I get some time I will try to add this.
Reply With Quote
  #3  
Old 09-23-2005, 03:29 AM
vsully vsully is offline
Member
 
Join Date: Sep 2004
Posts: 24
Send a message via AIM to vsully
Yea,

I'm using a custom .vzf head and tried <head>.clear(), but I still can't seem to get what I want.

What I need basically, is to stop the avatar from both morphing its face and speaking the .wav file. I've started the show with

words = <avatar>.speak
<avatar>.act(words)

but I'd like to have the ability to interrupt 'words' if necessary and reset and resync the avatar to speak 'words2' or the like.

On a different note, is it also possible to blend a .speak() with an animation, say walking? IE, the head is moving around with an animation, while the mouth is following the speach. I couldn't quite get that to work...

Thanks much.
Reply With Quote
  #4  
Old 09-23-2005, 10:31 AM
farshizzo farshizzo is offline
WorldViz Team Member
 
Join Date: Mar 2003
Posts: 2,849
Hi,

Yes, you can play to actions at the same time. When you add actions to objects you can specify which action pool to add it to. Action pools allow you to run multiple actions on objects in parallel. Please have a look in the documentation under the Actions section.

For example, lets say you wanted to perform a speak action and animation action in parallel:
Code:
#Create an action to play animation 3
anim = vizact.animation(3)

#Create an action to speak 'words.wav'
words = vizact.speak('words.wav')

#Play both actions on the avatar simultaneously
avatar.add(anim,1)
avatar.add(words,2)
To stop a current action from running you would do the following:
Code:
avatar.clear(viz.CURRENT_ACTION,pool)
This will stop any current actions being performed on the avatar on the specified pool. Currently, the speak action will not stop the sound from being played when it is cleared, so you will have to stop it manually:
Code:
viz.playsound('words.wav',viz.STOP)
On a side note, I had some free time and was able to create a version of the speak action which will keep the mouth movement in sync with the audio file. I tried it on a wav that was 5 minutes long and it seemed to work fine. I will include it in the next update to Vizard. If you wanted, I could supply you some code to get it working in the current version.
Reply With Quote
  #5  
Old 09-23-2005, 03:27 PM
vsully vsully is offline
Member
 
Join Date: Sep 2004
Posts: 24
Send a message via AIM to vsully
Thanks for the help -- I'll give that a shot and see what happens. I'd deffinitely like a copy of the audio sync code as well, it'll come in handy quite a lot here.
Reply With Quote
  #6  
Old 09-23-2005, 10:18 PM
farshizzo farshizzo is offline
WorldViz Team Member
 
Join Date: Mar 2003
Posts: 2,849
Hi,

I've attached a zip file to this post. It contains the files speak_sync.py and testAvatarSpeakSync.py. Place speak_sync.py in the [Vizard25]/plug-ins directory. Edit testAvatarSpeakSync.py and replace the wav file with your own file. Let me know if you have problems with it. Also, I'm assuming you have version 2.53 or above.
Attached Files
File Type: zip speak_sync.zip (1.5 KB, 1056 views)
Reply With Quote
  #7  
Old 10-25-2005, 12:38 AM
vsully vsully is offline
Member
 
Join Date: Sep 2004
Posts: 24
Send a message via AIM to vsully
Hi,

The previous is working great for me, save one small bug. I have four audio files, and by using viz.waittime() with the known file lengths I can wait until one audio is (presumably) finished before avatar.add()'ing the next _speak_sync action into the avatar.

Works perfect, save for rare runs of the program when the sound doesn't start but the program continues to execute normally. That is, the .add() call is executed but no audio is heard.

Is there a way to make sure a call to avatar.add() with a _speak_sync file will deffinitely play the associated audio file?

Thanks,
aaron
Reply With Quote
  #8  
Old 10-25-2005, 12:49 PM
farshizzo farshizzo is offline
WorldViz Team Member
 
Join Date: Mar 2003
Posts: 2,849
Hi,

Are you getting any error messages? Try waiting an extra second before adding the next speech just to make sure the audio is finished. What happens if you just add all the speeches to the avatar and let them play one after the other?
Reply With Quote
  #9  
Old 08-29-2006, 01:12 PM
betancourtb82 betancourtb82 is offline
Member
 
Join Date: Jan 2006
Posts: 103
More help.

I'm not sure what I'm doing wrong but I tried using the speak_sync module, but did not get the synchronization. I'm not sure if I'm doing something wrong, but I am not able to include the file I'm using due to it's size. The file is about 6 MB and lasts roughly about 2 minutes. Please let me know if there is any adjustments I need to make. The avatar continues to speak for about 2 minutes after the end of the clip.

Thanks
Reply With Quote
  #10  
Old 08-29-2006, 04:51 PM
Gladsomebeast Gladsomebeast is offline
Member
 
Join Date: Mar 2005
Location: Isla Vizta, CA
Posts: 397
Have you tryied using the avatar.speak() command?
__________________
Paul Elliott
WorldViz LLC
Reply With Quote
  #11  
Old 08-30-2006, 01:28 PM
betancourtb82 betancourtb82 is offline
Member
 
Join Date: Jan 2006
Posts: 103
Yes, I have tried both the regular speak command and the one shown below. They both keep the avatar's lips moving a lot after the wav clip stops.

Thanks

Code:
def Introduction(instructor):
 walk = instructor.walkto(0,0,10,1,260,2)
 #Clear all the avatar's actions, reposition it, and start the walk action
 instructor.clear(viz.ALL)
 instructor.visible(1)
 speech = instructor.speak(neutralpreanagram,.007)
 action1 = instructor.act(walk)
 action2 = instructor.turn(180)
 action3 = instructor.state(1)
 action4 = speech  
 action5 = instructor.walkto(8,0,10,1,260,2)
 action6 = instructor.walkto(8,0,-25,1,260,2)
 instructor.act(vizact.sequence(action1,action2,action3,action4,action5,action6))
Reply With Quote
  #12  
Old 08-30-2006, 02:53 PM
farshizzo farshizzo is offline
WorldViz Team Member
 
Join Date: Mar 2003
Posts: 2,849
Are you placing the speak_sync.py file in the vizard plug-ins folder? Also are you using vizact.speak_sync(..) to generate the action?
Reply With Quote
  #13  
Old 08-30-2006, 03:55 PM
betancourtb82 betancourtb82 is offline
Member
 
Join Date: Jan 2006
Posts: 103
I just moved the speak_sync.py document to the plugins folder (C:\Program Files\Vizard25\plug-ins). Here is the import statements and the current version of the code. I am still getting the same behavior. The avatar still keeps talking after the wav file is done. I can send you the wav file via email if that would help. Please give me an email address if that would help.

Thanks

Code:
import viz
from whrandom import randint,uniform
import math
import vizmat
import time
import vizinfo
from random import random
import speak_sync
Code:
def Instructions(instructor):
 walk = instructor.walkto(0,0,-2,1,260,2)
 #Clear all the avatar's actions, reposition it, and start the walk action
 instructor.clear(viz.ALL)
 speech = vizact.speak_sync('NeutralPostAnagram.wav')
 action1 = instructor.act(walk)
 action2 = instructor.turn(180)
 action3 = instructor.state(1)
 action4 = speech  
 action5 = instructor.walkto(5,0,-3)
 instructor.act(vizact.sequence(action1,action2,action3,action4,action5))
 #viz.killtimer(START_AVATAR)
Reply With Quote
  #14  
Old 09-05-2006, 03:29 PM
betancourtb82 betancourtb82 is offline
Member
 
Join Date: Jan 2006
Posts: 103
I figured out what happened. It turns out I was creating the .wav files in stereo instead of mono. I think that solved the problem.
Reply With Quote
  #15  
Old 08-23-2007, 05:19 PM
betancourtb82 betancourtb82 is offline
Member
 
Join Date: Jan 2006
Posts: 103
I just upgraded to Vizard 3.0 and am getting this error with a program that I previously ran in Vizard 2.5g. I know this is a bit general but I was wondering if anyone knew what was causing this. I placed the speak_sync in the plug-ins folder as I did with Vizard 2.5 but that didn't help.


Code:
Traceback (most recent call last):
** Load Time: 0.03 seconds
  File "<string>", line 11, in ?
  File "MoiraProjNeutralv2.py", line 17, in ?
    import speak_sync
ImportError: Bad magic number in speak_sync.pyc
Reply With Quote
  #16  
Old 08-23-2007, 05:24 PM
farshizzo farshizzo is offline
WorldViz Team Member
 
Join Date: Mar 2003
Posts: 2,849
You need to delete the speak_sync.pyc file in your directory, but leave the speak_sync.py file. Either way, I don't think you need this module anymore, as this option is built-in to Vizard 3.0. When creating the speak action just enable syncing like so:
Code:
speak = vizact.speak('file.wav',sync=True)
Reply With Quote
Reply

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 11:59 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2021, vBulletin Solutions, Inc.
Copyright 2002-2018 WorldViz LLC