delays with <avatar>.act and speech - WorldViz User Forum

vsully · #1 09-16-2005, 09:37 PM

I'm using <avatar>.speak() and <avatar>.act() to get an avatar talking. My problem is that with the sound files I have, (2-3 minute, 2-4 MB), the mouth movements will often end up with a ~5 sec lag by the end of the audio file.

Near 100% sync would be nice if possible, the other solution might involve clearing the face morphs when one audio file ends in order to begin and re-sync the next audio file, without the ~5 sec break in speech while the mouth wags in silence...

Towards the second solution, I'm using <avatar>.clear() with viz.ANIMATION and viz.CURRENT_ANIMATION, but it doesn't seem to be catching the mouth morphing.

Thanks

farshizzo · #2 09-19-2005, 03:16 PM

Hi,

Are you using the built-in head of the avatar or are you adding a vzf head? To clear the morph of a vzf head, you need to call the clear command on the face object of the avatar. If you are using the built-in head then you need to get the jaw bone of the avatar and reset its rotation to the following axis angle rotation [1,0,0,97.35].

However, it should be possible to keep the mouth movement synced. The code for the speak action is located in vizact.py. It currently uses the viz.playsound function to start the sound and doesn't do any resyncing afterwards. Instead of doing this you could add an audio object and periodically check what its current position is and resync the mouth movement with it. When I get some time I will try to add this.

vsully · #3 09-23-2005, 03:29 AM

Yea,

I'm using a custom .vzf head and tried <head>.clear(), but I still can't seem to get what I want.

What I need basically, is to stop the avatar from both morphing its face and speaking the .wav file. I've started the show with

words = <avatar>.speak
<avatar>.act(words)

but I'd like to have the ability to interrupt 'words' if necessary and reset and resync the avatar to speak 'words2' or the like.

On a different note, is it also possible to blend a .speak() with an animation, say walking? IE, the head is moving around with an animation, while the mouth is following the speach. I couldn't quite get that to work...

Thanks much.

farshizzo · #4 09-23-2005, 10:31 AM

Hi,

Yes, you can play to actions at the same time. When you add actions to objects you can specify which action pool to add it to. Action pools allow you to run multiple actions on objects in parallel. Please have a look in the documentation under the Actions section.

For example, lets say you wanted to perform a speak action and animation action in parallel:

Code:

#Create an action to play animation 3
anim = vizact.animation(3)

#Create an action to speak 'words.wav'
words = vizact.speak('words.wav')

#Play both actions on the avatar simultaneously
avatar.add(anim,1)
avatar.add(words,2)

To stop a current action from running you would do the following:

Code:

avatar.clear(viz.CURRENT_ACTION,pool)

This will stop any current actions being performed on the avatar on the specified pool. Currently, the speak action will not stop the sound from being played when it is cleared, so you will have to stop it manually:

Code:

viz.playsound('words.wav',viz.STOP)

On a side note, I had some free time and was able to create a version of the speak action which will keep the mouth movement in sync with the audio file. I tried it on a wav that was 5 minutes long and it seemed to work fine. I will include it in the next update to Vizard. If you wanted, I could supply you some code to get it working in the current version.

vsully · #5 09-23-2005, 03:27 PM

Thanks for the help -- I'll give that a shot and see what happens. I'd deffinitely like a copy of the audio sync code as well, it'll come in handy quite a lot here.

farshizzo · #6 09-23-2005, 10:18 PM

Hi,

I've attached a zip file to this post. It contains the files speak_sync.py and testAvatarSpeakSync.py. Place speak_sync.py in the [Vizard25]/plug-ins directory. Edit testAvatarSpeakSync.py and replace the wav file with your own file. Let me know if you have problems with it. Also, I'm assuming you have version 2.53 or above.

vsully · #7 10-25-2005, 12:38 AM

Hi,

The previous is working great for me, save one small bug. I have four audio files, and by using viz.waittime() with the known file lengths I can wait until one audio is (presumably) finished before avatar.add()'ing the next _speak_sync action into the avatar.

Works perfect, save for rare runs of the program when the sound doesn't start but the program continues to execute normally. That is, the .add() call is executed but no audio is heard.

Is there a way to make sure a call to avatar.add() with a _speak_sync file will deffinitely play the associated audio file?

Thanks,
aaron

farshizzo · #8 10-25-2005, 12:49 PM

Hi,

Are you getting any error messages? Try waiting an extra second before adding the next speech just to make sure the audio is finished. What happens if you just add all the speeches to the avatar and let them play one after the other?

betancourtb82 · #9 08-29-2006, 01:12 PM

I'm not sure what I'm doing wrong but I tried using the speak_sync module, but did not get the synchronization. I'm not sure if I'm doing something wrong, but I am not able to include the file I'm using due to it's size. The file is about 6 MB and lasts roughly about 2 minutes. Please let me know if there is any adjustments I need to make. The avatar continues to speak for about 2 minutes after the end of the clip.

Thanks

Gladsomebeast · #10 08-29-2006, 04:51 PM

Have you tryied using the avatar.speak() command?

betancourtb82 · #11 08-30-2006, 01:28 PM

Yes, I have tried both the regular speak command and the one shown below. They both keep the avatar's lips moving a lot after the wav clip stops.

Thanks

Code:

def Introduction(instructor):
 walk = instructor.walkto(0,0,10,1,260,2)
 #Clear all the avatar's actions, reposition it, and start the walk action
 instructor.clear(viz.ALL)
 instructor.visible(1)
 speech = instructor.speak(neutralpreanagram,.007)
 action1 = instructor.act(walk)
 action2 = instructor.turn(180)
 action3 = instructor.state(1)
 action4 = speech  
 action5 = instructor.walkto(8,0,10,1,260,2)
 action6 = instructor.walkto(8,0,-25,1,260,2)
 instructor.act(vizact.sequence(action1,action2,action3,action4,action5,action6))

farshizzo · #12 08-30-2006, 02:53 PM

Are you placing the speak_sync.py file in the vizard plug-ins folder? Also are you using vizact.speak_sync(..) to generate the action?

betancourtb82 · #13 08-30-2006, 03:55 PM

I just moved the speak_sync.py document to the plugins folder (C:\Program Files\Vizard25\plug-ins). Here is the import statements and the current version of the code. I am still getting the same behavior. The avatar still keeps talking after the wav file is done. I can send you the wav file via email if that would help. Please give me an email address if that would help.

Thanks

Code:

import viz
from whrandom import randint,uniform
import math
import vizmat
import time
import vizinfo
from random import random
import speak_sync

Code:

def Instructions(instructor):
 walk = instructor.walkto(0,0,-2,1,260,2)
 #Clear all the avatar's actions, reposition it, and start the walk action
 instructor.clear(viz.ALL)
 speech = vizact.speak_sync('NeutralPostAnagram.wav')
 action1 = instructor.act(walk)
 action2 = instructor.turn(180)
 action3 = instructor.state(1)
 action4 = speech  
 action5 = instructor.walkto(5,0,-3)
 instructor.act(vizact.sequence(action1,action2,action3,action4,action5))
 #viz.killtimer(START_AVATAR)

betancourtb82 · #14 09-05-2006, 03:29 PM

I figured out what happened. It turns out I was creating the .wav files in stereo instead of mono. I think that solved the problem.

betancourtb82 · #15 08-23-2007, 05:19 PM

I just upgraded to Vizard 3.0 and am getting this error with a program that I previously ran in Vizard 2.5g. I know this is a bit general but I was wondering if anyone knew what was causing this. I placed the speak_sync in the plug-ins folder as I did with Vizard 2.5 but that didn't help.

Code:

Traceback (most recent call last):
** Load Time: 0.03 seconds
  File "<string>", line 11, in ?
  File "MoiraProjNeutralv2.py", line 17, in ?
    import speak_sync
ImportError: Bad magic number in speak_sync.pyc

farshizzo · #16 08-23-2007, 05:24 PM

You need to delete the speak_sync.pyc file in your directory, but leave the speak_sync.py file. Either way, I don't think you need this module anymore, as this option is built-in to Vizard 3.0. When creating the speak action just enable syncing like so:

Code:

speak = vizact.speak('file.wav',sync=True)

Thread Tools
Show Printable Version Email this Page
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: