WorldViz User Forum  

Go Back   WorldViz User Forum > Vizard

Reply
 
Thread Tools Rate Thread Display Modes
  #1  
Old 08-10-2010, 10:18 AM
Philippe Philippe is offline
Member
 
Join Date: Jul 2010
Posts: 12
vizact.speak

Hi,

When using vizact.speak, it is very difficult to have the Avatar speaking in a natural manner. When setting the two following parameters:

Scale =
Threshold =

we managed to set "Scale" in a satisfactory manner, but we had troubles finding something appropriate for the "Threshold". If we set it high, there were parts of the speech where the Avatar did not open its lips. If we set it low, then there were non natural movements of the lips appearing when the lips should hardly move or not move at all.

What is the best way to make the Avatar speaking in a natural manner? Is the use of vizard.speak method the only way, or is there a more sophisticated method? Which is the best morph to use? I guess the quality of the recording of the sound file is important, but to what extent? Are there any tips concerning the sound file recording (speaking loudly in the microphone, use a certain type of microphone, format of the audio file, etc.)? We noticed that some types of voice gave better results than others. What is the current usage – taking some samples, and then trying different morphs to see how it best fits?
Even when the movement was acceptable and gave some approximation of a real environment, it still was quite mechanical – all what we could control was to what extent the mouth opens (Scale) and the sensitivity to sound amplitude (Threshold). Are there other methods giving more control on the movement of the lips - not only open/close and amplitude? Is there some utility which can enable us to match the movement of the lips to the International Phonetic Alphabet? Finally: are there plans to improve vizard.speak method in the foreseeable future?

Thanks.
__________________
Thanks,

Philippe
Reply With Quote
  #2  
Old 08-10-2010, 02:37 PM
masaki masaki is offline
Member
 
Join Date: Jan 2008
Posts: 63
Hi Philippe,
Aside from going into a 3d software like 3D Studio Max and manually creating an animation for the mouth movements specifically for your sound file, there isn't a more sophisticated method other than vizact.speak (for now). Internally, we're discussing how to improve many avatar features - and one of the features we want to beef up is the speak method.
The best you can do for now is to have a very clean recording and to tweak the scale and threshold parameters. I have some pointers for getting a clean sound file:
1) if you can avoid it, do not use a pc mic connected to the mic line of your soundcard.
2). Get a real microphone (e.g. Shure SM58 for $100 would be top notch for this) and boost the signal using a preamp, multitracker, etc, and feed the signal to the line in of your sound card (*not* the mic line).
3) use a pop filter OR (this is important especially if you're using a pc mic) speak into the microphone at an angle instead of head on. This is to reduce the "pops" in the sound file you get from plosives (words with p's, b's, etc like the word "pop"). When you speak these words a burst of air hits the microphone if you are head on, and it makes the sound "clip" (distortion caused by maxing out the input level).
4) finally, use an audio editing software (e.g. Audacity is a freeware) to clean up the sound. Use high/low pass filters to get rid of the low hums and rumbles and also the high pitched hisses. You can also use parametric eq's to boost the mid range (where most voices fall). Filters like pop-filters, click-filters, etc should be helpful as well. Basically you want a loud and clean sound file without any noise or clipping. That will yield the best lip movement.

Best,
Masaki
Reply With Quote
  #3  
Old 08-10-2010, 06:36 PM
Deltcho Deltcho is offline
Member
 
Join Date: Sep 2007
Posts: 18
Hi Philippe,

I've been working with making avatars talk as well. Maybe I can share some of the things that I've found to work.

As masaki has already suggested, using audio editing software is key. I've personally had *a lot* of success using a standard noise cancellation PC mic (ZALMAN ZM-MIC -- costs about $5-10). The mic clips onto the collar of your shirt.

Here's the process I came up with:

1) Record the audio using proper audio software (I personally use Acoustica but I'm sure other ones work just as well.) When speaking, speak in a *natural tone* as if you were talking to someone else in the room.

2) Normalize volume in the speech file to 20 db (all recorded files should be normalized to this for best results with my settings.)

3) Run a noise reduction filter on it.

4) Export the file as a mono 16-bit PCM file (very important for proper syncing).

5) Use a scale value of 0.028 and a threshold of 0.0001

I've currently got 4 pages of dialogue lip synced perfectly using this method. I also recommend tweaking vizact.py to sync the lips more often (described in my other post.)

I'll try and post a video demonstrating the end result if I have time.
Reply With Quote
  #4  
Old 08-12-2010, 01:01 AM
Philippe Philippe is offline
Member
 
Join Date: Jul 2010
Posts: 12
Hi Masaki, Hi Deltcho,

Thanks a lot for your detailed answer! This is extremely valuable information for us, since until now, we were not really happy with the results, although we tried very hard to put the right parameters. I am sure the improved quality of the sound file using the techniques, the tools and the material you recommended will improve drastically the animation.

Thanks again, I really appreciate your tremendous help.
__________________
Thanks,

Philippe
Reply With Quote
Reply

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
vizact.speak mjabon Vizard 1 07-07-2008 02:39 PM


All times are GMT -7. The time now is 03:23 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
Copyright 2002-2023 WorldViz LLC