Skinning with VVVV’s character animation nodes
After having explained how to create skeletons for 3D characters and animating them in VVVV, this quasi-article shows how to use VVVV’s character animation nodes in combination with software and hardware skinning methods.
The prevous quasi-article introduces some nodes which were designed to create skeletal animation in VVVV. However, the article didn’t describe how those nodes can actually deform 3D geometry. So this article shows an example of how to build and skin a kind of abstract character model based on a skeleton dynamically.
UPDATE: I uploaded an extended version of the patch described below, which also shows some parts from the previous article. You can download it here: walkingboxes.rar.
Some skinning theory
Skinning describes the process of binding a 3D geometry to a skeleton. The skeleton acts as some kind of abstraction layer over the 3D geometry: instead of moving single vertices, the animator can move joints - the 3D vertices are moved accordingly.
Besides X,Y,Z coordinates, every 3D vertex can have further data assigned, e.g. bind indices and skin weights. Those measures describe, to which extent a vertex is influenced by a certain joint. The bind index is the index of the influencing joint, and the skin weight is the amount of influence, the joint has on a vertex.
If each vertex of a model is influenced by only one joint to full extent, we call this rigid skinning. If multiple joints influence one vertex, we speak of soft skinning or smooth skinning. As you might have guessed, the results here are softer. And smoother, as well. If you are working with vertex buffers and hardware skinning, the number of influencing joints per vertex is limited to 4.
Building some character
We already talked about creating skeletons, so for this, we take a humanoid skeleton as granted. The image below shows the skeletal structure, and the global joint positions represented by quads in the renderer.
We can use the joint positions to create some fancy 3D geometry. For this example, I’m going to build one from some boxes. I’m not going into detail on how I did this, because it’s not looking that awesome anyway, so we will take this as granted as well
The only way to create perfect skin weights still is manually doing it in a modelling package. This would be the way to go, if we created a model e.g. in Maya and imported it into VVVV using the COLLADA format. Since we built our geometry in VVVV, we need a way to create skin weights in VVVV as well. There are algorithms which automatically calculate skin weights. The most trivial of all is fully assigning a vertex to its nearest joint, which of course results in a rigid skinned mesh. Exaclty this is done by the node
AutoSkinWeights (Skeleton). It takes vertex coordinates and a skeleton, and computes simple skin weights. Of course, there are smarter algorithms, like using heat maps, which aren’t implemented yet.
Since our mesh is now officially skinned, we can go on and actually deform it. The image above already contains the Skinning.fx shader effect. This effect does the calculation of the deformed vertices based on the joint transformations of the skeleton. All we have to do, is using
GetJointTransform (Skeleton) on a posing skeleton, to get all the joints’ compiled transformations, and pass it on to the Skinning effect. The image below shows, how this would look like.
The inverse bind pose transformation
There’s still one thing in the image above, that has to be clarified: the 2nd input of
GetJointTransform (Skeleton), which gets the inverse of the initial, unanimated joint transformations. This is, what in literature is referred to as the Inverse Bind Pose Transformation (IBP). The IBP holds information about the bind pose - the skeleton’s pose in the moment, the skin weights of a geometry are defined.
In our example, the skeleton’s initial pose, and the model’s initial pose match up, so the IPB is just the inverse of the “still standing” skeleton. Sometimes, this is not the case, e.g. if you use the same skeleton for different meshes. If you are using a mesh you imported from an external source, the IBP should be delivered by the importer.
A word on software skinning
Although using a shader effect for deforming geometry is definitly the common way to go, in certain cases you have to calculate the vertex positions on the CPU. This is, e.g. if your character rig exceeds the current hardware limits of 60 joints or 4 influences, or if you need the transformed vertex positions for further processing. In that case you can use the
SkinDeformer (Skeleton), which basically does the same calculation as the Skinning vertex shader effect. However, instead of GPU calculation,
SkinDeformer (Skeleton) works on the CPU, and returns the deformed vertices as vector spread, which then can run into a vertex buffer, or can be used however you want.
What to do in the future
Besides being very slow,
AutoSkinWeights (Skeleton) only performs rigid skinning, which results in un-smooth deformation in joint regions. This might be ok for prototyping and in the cases, the result should look noisy anyway. But for more advanced projects,
AutoSkinWeights (Skeleton) should also provide a soft skinning algorithm. Using heat maps as proposed in section 4 of Automatic Rigging and Animation of 3D characters looks promising.