Problems with 3DMeNow Avatar Heads

[kim_swinth]

Hi Farshid,

I'm having several problems with avatars that I'm hoping you can help me with.

First, I'm having a problem with avatar heads created with 3DMeNow. Specifically, for several of the female heads I've made, small pieces of the mouth geometry seem to be "poking through the cheek" whenever the avatar smiles. Any idea how to fix that?

Second, I'm trying to work with some functions that Andy and Alex Dimov wrote which allows you to remove the eyes from the 3DMeNow head (imported into Vizard) so that the eyes can be animated. I can get the eye geometry to be removed okay, but the texture map from the face seems to be getting applied to the eyes rather than the texture map for the teeth, eyes, and tongue that gets created when you export a 3DMeNow head. Is there a way that I can specify what texture map gets applied there?

Finally, do you have any better documentation on the avstu commands? I've looked at what's in the Vizard documentation, but I've also noticed that there are a lot of other commands getting called from the avatar.py script and the avatar3d.py script that Andy and Alex wrote. It would be helpful to know what those commands are doing so I can modify some of the functions for a project I'm working on.

Let me know if you need more details about any of these questions. I had written a much longer, more detailed message, but it seems to have gotten lost in cyberspace

Thanks for your help.

Kim

[farshizzo]

Hi Kim,

Try lowering the percentage of the smile in order to stop polygons from poking through the cheek. The smile command simply makes the avatar smile at 100%. Try the following:

avatar.setExpression(601, 0.8)

This will make the avatar smile at 80%

I've never worked with animating eyes before, so I really don't know how to get that working. But from my understanding all it does is move the eye texture around, it doesn't actually move the geometry. Have you tried using it without changing the geometry? Andy knows more about this, I'll see if he knows how to get it working.

For some reason the avstu commands were removed from the documentation. I'll see if I can get you a copy of that. Is there a specific command you are not sure of? Most of the commands in Avatar.py have comments explaining what they do, have you already looked at these?

[kim_swinth]

Hi Farshid,

I'll try making the avatar smile at a lower % to see if that helps. Although I have already tried really cranking down the factor in the filterHead.py script which helped some, but I had brought the emotional expressions down about as low as I could tolerate (they are pretty subtle now as it is). I'm worried that going even lower on the smile% will produce an unnoticable smile, but it's worth a try. If that doesn't work, do you have any suggestions. I'm not quite sure how the filterHead script works, but I was wondering if there was a way to minimize those particular polygons somehow.

I actually tried taking one of my heads into 3D Studio Max and trying to pull in the polygons, but that didn't seem to do much. So it seems that it's not really where the polygons are relative to the head in the model (although I guess it could be), but rather, it seems to be what is happening when I issue a smile command. So let me know if you think of anything else to help with this problem.

Regarding the eye movements, it's very possible that it's only moving the texture, which is just fine with me. The real problem seems to be that it's applying the wrong texture map to the eyes. According to Alex, this used to work just fine under VRUT, but it doesn't now under Vizard. Any ideas about what might have changed that would have affected this? Also, any idea about where it is looking for the texture for the eyes. If I can change the .jpg designation it's looking for, it might solve the problem. I just can't find where's it's finding that information.

Finally, as for the avstu commands, the documentation I have only tells what commands 1-8 and 100-500 do. But the avatar.py script uses a bunch of others (the one's I remember were 12, 1000, 600, and maybe some other ones in the hundreds). So it appears that there is more functionality available, but I can't find that it's documented anywhere. So anything you can dig up would be really apprciated.

Thanks

Kim

[kim_swinth]

Hi Farshid,

One other thing....

Is there anything that shows all the setExpression commands? In your last reply, you indicated to use the avatar.setExpression(601, 0.8). Is there anything that shows the different setExpression commands that can be used? I only know about the Expression list in the documentation, but given that we do a lot with social interaction, the expressions are something we'll be using a lot. So it would be really helpful to know the different functionality for controlling expressions (like the numbers for the different expression and changable expressions -- like being able to only smile at 80%). That command alone will be really helpful in the current study I'm designing. But it would be great to know about the other commands too.

Thanks!

Kim

[farshizzo]

Hi Kim,

I'm attaching a script which I created back in my recveb days. Run the script and it will prompt you to browse to a face file. It will then change the intensity of all the facial expressions by a given factor. The following line should be near the beginning of the script:

FACTOR = 0.6

This will cause the script to decrease the intensity of the facial expressions by 60%. Change this value to whatever you want.

Here's a list of all the facial expression numbers which you can pass to avatar.setExpression:
[list=1][*] Mouth Open[*] Frown[*] Blink[*] Jawdrop[*] Raise Left Eyebrow (negative value will lower eyebrow)[*] Raise Right Eyebrow (negative value will lower eyebrow)[*] Smile[*] Pout[*] Widen Eyes[*] Big Smile[*] Squint[/list=1]

To find what texture file it is looking for, open up your face file and do a search for "jpg". You should find the texture file for the face and the texture file for the mouth. Look in the Vizard\avatar directory. There should be a file "mouth_texture.jpg", I think this is the default file.

I looked through the code of the avstu plugin and I found what the following commands do that you requested:

12 - Toggles whether the avatar can move its eyes
13 - Moves the avatars eyes by a given horizontal and vertical value
1000 - Resets the avatars facial expressions to neutral, this is what avatar.normal uses
601 -> 611 - This sets an individual facial expression to a given percent. This is what setExpression does, so you really don't need to use this.

Hope this helps, let me know if you have any other problems.

[kim_swinth]

Hi Farshid,

The command you suggested, avatar.setExpression(601, .80) doesn't seem to do anything. Is there a module I need to import to get this to work? If I do avatar.smile(), my avatar smiles (but with the cheek poke-through problem). But the setExpression doesn't do anything.

Any ideas?

Kim

[kim_swinth]

Hi Farshid,

Please disregard my last message. I found the problem. The command that needs to be sent is:

avarar.setExpression(7, .8) not avatar.setExpression(601, 1.5). Now she smiles, and if I take the percentage down to 60%, I can avoid the poking-through problem. But it does give a pretty subtle smile (more subtle than I'd really like). So if you think of another cause of this problem and anything I can do about it, please let me know.

Kim

[farshizzo]

Hi Kim,

The setExpression command takes a number from 1-11. Those numbers correspond to the expressions I mentioned in the previous post. The setExpression command takes this number, adds it to 600, and then sends it to the avstu plugin. So if you want your avatar to Frown 50% you would do:

avatar.setExpression(2,0.5)

Hope that clears it up!

[farshizzo]

Fine then, I guess you don't need my help

You might need to remodel the head in 3DMeNow and make sure the cheek polygons are pushed out farther so that the teeth don't poke through. As far as I can tell, this is the only other way to fix the problem, other than removing the teeth, but I'm assuming you want the teeth to be rendered.

[kim_swinth]

Hi Farshid,

I just thought I give you an update about my teeth poking through the cheek problem in case anyone has this problem. According to Andy, the problem most likely is occuring because when the normalizehead.py script is run, the delta defaults used were from a particular (male) head and that if the 3DMeNow head you are filtering is a lot different, the deltas may be wrong. Andy said it's possible to create a customized delta default file for my avatar, but it sounds pretty involved. I got things working "okay" for this study, but this is something you guys might want to think about a little for the future if you're going to support 3DMeNow heads. It would be great if a custom delta defaults file could be created for each head -- then the expressions would look more consistent across different heads. But this also helps explain why expressions tend to look different on different heads.

Thanks again for you help on this.

Kim