making a talking and imitating avatar

[marc van gaal]

I have used the avatar.face command to replace the head of our avatars with a head we morphed so it can speak. This wasn't much trouble and went smoothly.

We are now trying to let the avatar imitate headmovements made with a HeadMountedDisplay. And this is where we run into some trouble.

The movements made by the HMD are to be repeated by the neckbone so the avatar makes the same headmovements. We first found out that the weights we assigned to the head were not translated to vizard after using peoplemaker. This resulted in the head, neck and shoulders moving as one object. The result was that when turning the head to the left the shoulders also turned breaking through the body texture.

We therefore tried to tie the neck to the body to keep it's original weights. We then exported a face without a neck to peoplemaker to add the morph for talking. We replaced the original face in viard with morph face and this time the avatar could speak and move his head in a natural manner. However, for some reason the face was slightly parted from the neck and we could not get vizard to place it exactly on top of the neck.

After reading the peoplemaker manual again (read the ##*&&# manual!! I know) we discovered peoplemaker should be able to compute neck verticels. We tried to use this on our earlier face that still had a neck and then tried to implement it in vizard. But again, things went wrong.

At the moment we have no idea how to fix this, so if anyone has been in this situation before or knows what to do.... I'd appreciate the help.


Marc

[Gladsomebeast]

With the neck included head, try roatating the "skel_Head" bone instead of the neck bone. I think the shoulder bones are children of the neck bone.

[marc van gaal]

Maybe I left our a rather improtant detail in my story. The avatar I am using is not a standard vizard avatar. We use a call3d avatar and head. This may change things a little.

Marc

[rdotsch]

Hi!

I'm working with Marc on this project. To rephrase, the problem is as follows:

For an experiment we need to have an avatar which imitates a user's head movements, while speaking. To this end, we create our own avatar.

1) We exported the avatar to Cal3d, which worked fine. We were able to have him imitate our head movements.

2) We also were able to replace the head with a head we made in people maker, so the avatar would be able to speak. The head mesh had a neck already attached to it, the body didn't have a neck. This worked fine.

3) But we got problems when we want to combine the imitation with the talking. People maker made the neck vertices lose the weights we assigned to them in Maya/3ds Max.

4) We tried to solve this by cutting the head and the neck into two seperate meshes, keeping the neck attached to the body, so it'll keep the weights in Cal3d. Then, in people maker we selected the neck vertices before creating the morphs. This didn't work out, because, depending on to which bone we try to clamp the newly created head, either the selected neck vertices are pulled down, but not attached to the body mesh (the neck), or we get some empty space left over between the neck and the head.

5) Rotating the head bone instead of the neck bone will not solve the problem, because the problem is already there before we even rotate anything (the open space between the neck and the head).

What are we doing wrong?

If you want, we can post our cal3d avatar, and the heads we created in people maker.

Also, it would help if we would know what the clamp and the head variables mean with the avatar.face() function. It seems that the clamp variable needs to be a bone, but which bone then (as we already mentioned, we don't use the avatars that come with vizard)?

Thanks in advance for your help,

Ron

[farshizzo]

Also, it would help if we would know what the clamp and the head variables mean with the avatar.face() function. It seems that the clamp variable needs to be a bone, but which bone then (as we already mentioned, we don't use the avatars that come with vizard)?

The head parameter of the avatar.face() function needs to be the bone you want to attach your head to. The clamp parameter should be the bone you want the neck vertices to be clamped to. Vizard will transform the neck vertices of your PeopleMaker face into the coordinate system of the clamp bone. The default clamp bone is skel_Neck. If your neck bone is named differently, then you will need to pass in the name to this parameter.

[marc van gaal]

We have tried playing around with the head and clamp functions, but without any succes.
We still face two important problems.

1. the original avatar face is replaced by a morph face, but the original face does not dissapear anymore. Our code worked fine before when we only made a talking head. But now that we are trying to make a talking and moving head the original face stays visible.

2. Vizard will not put the replacement head on the right position. No matter what bone we refer to the face is always positioned to far away from the body. We assigned all the neck verticles in people maker hoping that this would help in positioning the face, but it didnt.


We have been working on this for days now but we can't solve these last two problems. If anyone has any ideas, we welcome all suggestions.
If anyone is willing to take a lookat the avatar and the morph face you can dowload it using the following link:

www.socsci.ru.nl/bsi/marc/avatar.zip


Marc

[farshizzo]

I tried to get your models working and I ran into a couple problems.

1) Your head mesh is split up into submeshes, so turning off the visibility doesn't work. This is a bug in Vizard and has been corrected.

2) Your PeopleMaker head is too small and facing the wrong direction. I was able to fix this by loading the face into PeopleMaker and changing the rotation/scale.

3) The neck vertices on your face are defined too high up on the face. You should consider adding the neck to the face and moving the neck vertices down. Since Vizard will stretch the neck vertices to the location of the clamp bone, this will look strange with the face you provided.

[marc van gaal]

Thanks for the good suggestions.
I'm working on the neck verticles at the moment.


What I don't understand is how the peoplemaker head can be to small. I used the button to scale the head to vizard standards, so how could it be to small? I don't see anything wrong with in vizard as well, the head seems to have the same size as the original head.

Marc

[farshizzo]

Hi,

You are right, the PeopleMaker head is the right size. The problem is that your avatar is too big. Try scaling down the size of the avatar using the scale property in the cfg file.

[marc van gaal]

Thanks for all the suggestions. We finally managed to solve all the problems and it looks great now!

Marc