WorldViz User Forum  

Go Back   WorldViz User Forum > Vizard

Reply
 
Thread Tools Rate Thread Display Modes
  #1  
Old 02-21-2013, 12:39 AM
Frank Verberne Frank Verberne is offline
Member
 
Join Date: Mar 2008
Location: Netherlands
Posts: 148
HD complete characters speaking naturally

We're thinking of buying the HD complete character set. One very interesting features of these characters would be the facial expressions. I'm especially interested in making the characters speak more naturally (the lipflapping of vizact.speak() is far from optimal for conversational agents). Is it possible to let the HD characters speak with phonemes (see http://www.karisproductions.com/wp-c...neme-Chart.jpg)? If so, how would you extract phonemes from a wav-file (or a sound in another format), so that the HD characters gets the information which lip movements to make to speak realistically? Would you need the Microsoft Speech API for this?

Most important question: could we test this with the HD complete character that comes with Vizard4?
Reply With Quote
  #2  
Old 02-27-2013, 11:27 AM
Veleno Veleno is offline
WorldViz Team Member
 
Join Date: Sep 2005
Posts: 148
Hi Frank,

Our characters are not currently set up with phonemes included, though these could be created either as morph targets or with the included facial bones.

Third-party programs exist for extracting phonemes from audio files, though I don't have any experience with them. One I have seen recommended is CMUSphinx. I also wouldn't be surprised if a python library exists that allows phonemes to be detected in real time.
Reply With Quote
  #3  
Old 02-27-2013, 12:27 PM
Veleno Veleno is offline
WorldViz Team Member
 
Join Date: Sep 2005
Posts: 148
To clarify, the VCCHD set has expressions and phoneme pose files (both in .bip and .cpy) that can be loaded onto the skeletons in Max. As of this moment they are not currently part of the exported versions of the avatars. It may be worth adding them to the DVD build.

The phonemes included as pose files are:
Quote:
- frame_00: NEUTRAL
- frame_01: A; K; I
- frame_02: Ah
- frame_03: B; M; P
- frame_04: C; D; K; N
- frame_05: Ch; Sh
- frame_06: E
- frame_07: Eh
- frame_08: F; V; W
- frame_09: G; J
- frame_10: G; J; R
- frame_11: L
- frame_12: N
- frame_13: O
- frame_14: S; X; Z
- frame_15: T
- frame_16: Th
- frame_17: U; W; Y
The expressions included as pose files are:
Quote:
- frame_00: NEUTRAL
- frame_01: Angry_01
- frame_02: Angry_02
- frame_03: Angry_03
- frame_04: Angry_04
- frame_05: Bored/Disgusted
- frame_06: Disgusted
- frame_07: Eye_left_closed
- frame_08: Eye_right_closed
- frame_09: Facial_shrug
- frame_10: Friendly
- frame_11: Frightened_01
- frame_12: Frightened_02
- frame_13: Grin
- frame_14: Grin_left
- frame_15: Grin_right
- frame_16: Kiss
- frame_17: Laugh_01
- frame_18: Laugh_02
- frame_19: Laugh_03
- frame_20: Narrow
- frame_21: Pain
- frame_22: Puh
- frame_23: Sad_01
- frame_24: Sad_02
- frame_25: Sad_03
- frame_26: Sad_04
- frame_27: Shout
- frame_28: Smile_01
- frame_29: Smile_02
- frame_30: Smile_03
- frame_31: Surprised
Reply With Quote
  #4  
Old 02-28-2013, 02:20 AM
Frank Verberne Frank Verberne is offline
Member
 
Join Date: Mar 2008
Location: Netherlands
Posts: 148
Hi Valeno,

Thanks for the response! It sounds like (no pun intended) the VCCHD set has all the ingredients to create a naturally speaking avatar, when phonemes are extracted from a .wav file. Too bad none of them are included with the standard character that comes with Vizard. I noticed that the HD example character does not have any morphs

It would be a good idea to include the different poses in the exported versions of the avatars on the DVD build! I also work with FaceGen, which includes similar morphs for the phonemes, so I will test those. If I can make those phonemes work, I think we can also get the VCCHD characters to speak properly.

Regards,
Frank
Reply With Quote
Reply

Tags
hd complete characters, microsoft speech api, phonemes, speak

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Complete Characters HD and speech Enlil Vizard 1 06-01-2011 06:16 PM
Complete characters and heads Vkathrynr Vizard 1 08-27-2009 11:09 AM
adding 3D heads to complete characters Vkathrynr Vizard 2 08-19-2009 09:16 PM
How to add individual faces onto the complete characters used in Vizard? VDong Vizard 3 11-12-2008 11:05 AM
How to cut out the heads of the complete characters? VDong Vizard 2 09-26-2008 06:27 PM


All times are GMT -7. The time now is 09:33 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Copyright 2002-2018 WorldViz LLC