Advertisement

Didn't quite understand the inverse bind pos matrix

Started by September 19, 2018 08:51 PM
9 comments, last by Pilpel 6 years, 2 months ago

The way I'm doing things right now is:

For static meshes, I calculate a "toRootTransformation" 4x4 matrix for every node, based on the hierarchy of the nodes. Then for every node that contains a mesh, I render the mesh with a shader looking like this:


    #version 430
    
    uniform mat4 ProjectionMatrix;
    uniform mat4 CameraMatrix;
    uniform mat4 ModelMatrix;
    
    layout(location = 0) in vec3 vertex;
    layout(location = 1) in vec2 uvs;
    layout(location = 2) in vec3 normal;
    
    out vec2 fsUvs;
    out vec3 fsNormal;
    
    void main()
    {
        gl_Position = ProjectionMatrix * CameraMatrix * ModelMatrix * vec4(vertex, 1.0);
    
        fsUvs = uvs;
        fsNormal = normal;
    }

Before that I set the ModelMatrix uniform to that toRootTransformation.

For skinning, I upload the bone matrix array uniform to the shader, calculate each vertex position based on it, and multiply by the same "toRootTransformation" as before.
Here's the main part of the vertex shader:


    void main()
    {
        vec4 final = vec4(0);
    
        for(int i = 0; i < NUM_INFLUENCE_BONES; i++)
        {
            vec4 v = Bones[boneIndices[i]] * vec4(vertex, 1.0) * vertexWeights[i];
            final += v;
        }
        
        gl_Position = ProjectionMatrix * CameraMatrix * ModelMatrix * final;
    }

Every bone matrix is a multiplication of the bone's inverseBindPos matrix and the matrix that is the product of all the rotations/translations of the bone at the current time, based on the node hierarchy. In code:


    void Bone::calculateFinalMatrix() { //matrix for the shader
        _finalMatrix = _animationMatrix * _inverseBindPosMatrix; //_animationMatrix is already calculated by AnimationController
    }

To be honest this is the product of like a week of struggling to get skinning done, and I'm not sure if I'm doing it right. (please correct me if not)

The question that I ask myself is why do I need to multiply each bone by the inverseBindPos first. It transforms each vertex from world to local space, before multiplying by the animation matrix. But the thing is that the vertex isn't in world space to begin with, but in local space. So basically we're going from local space to local space (then animating) which doesn't make sense.

This can be slightly confusing, you might want to refer to this thread:

https://www.gamedev.net/forums/topic/694891-use-assimp-for-skeletal-animation-help/

Afaik, if you had a robot with fixed links and no weighted vertices (i.e. each vertex is hard assigned to a bone) you could load it in with each body part in 'bone space' (i.e. root at the origin and pointing out along the default axis). Then you apply the calculated bone transformation to these verts and you are done.

However the reason we load the mesh in the 'rest pose' is because many of the vertices around joints will not be linked to just one bone but two or more, i.e. they have to go through the whole transformation process TWICE (or more), and thus they don't have just 1 bone space.

So the inverse rest pose transform is to get a vertex from the rest pose position to bone space FOR THAT PARTICULAR BONE.

Consider a forearm vertex in between forearm and bicep:

  1. Backtransform from rest pose A to 'forearm space', apply forearm animation matrix -> vertex position B
  2. Backtransform from rest pose A to 'upperarm space', apply upperarm animation matrix -> vertex position C
  3. Weighted blend between B and C to get final skinned position D

So overall, as I understand it, the whole rest pose / inverse rest pose matrix thing is to allow skinning to work correctly, not bones animation, if that makes sense.

Advertisement

You don't want local space, you want bone head space. All rotations are around the pivot point for that bone, not the pelvic region or wherever local space 0,0,0 is. PS if you can get animation up and running in a week you are doing well.

Can you elaborate about bone space? How does it differ from local space?

@TeaTreeTim But from what I understand 0,0,0 in local space 

@lawnjelly "So the inverse rest pose transform is to get a vertex from the rest pose position to bone space FOR THAT PARTICULAR BONE." But the vertex starts in local space, which means it's not in rest pose.. no? A vertex gets to rest pose only after I multiply it by the node transformation matrix (the toRootTransformation matrix in my post)

9 minutes ago, Pilpel said:

Can you elaborate about bone space? How does it differ from local space?

@TeaTreeTim But from what I understand 0,0,0 in local space 

@lawnjelly "So the inverse rest pose transform is to get a vertex from the rest pose position to bone space FOR THAT PARTICULAR BONE." But the vertex starts in local space, which means it's not in rest pose.. no? A vertex gets to rest pose only after I multiply it by the node transformation matrix (the toRootTransformation matrix in my post)

Local space is quite a vague term, and given that what you have doesn't seem to be working, it is better that we are explicit in descriptions.

Bone space as I think TeaTreeTim and myself are referring to is the space where the origin of the bone around which rotations take place is at 0, 0, 0. It will have the bone heading out along one particular axis, the default axis, which will depend on your exporter / convention.

The vertex should NOT start in 'local space' if you mean by that bone space. It should start in the rest pose position, i.e. if you drew the starting vertex positions with no transformation applied, you would get a character in the t-pose.

The reason the vertices are defined (start from) in terms of the rest pose and not in terms of bonespace, is as I explained earlier, a vertex is not necessarily a child of only one bone, it can be influenced by several bones, and is not like other models in a scene graph.

To get it working the first step would be to draw the skin with no transform and see what happens and see what you are working with. What method did you use to get export the bones / skin?

I think I understand but I'm not sure. Please don't give up on me ?

First off my code is running well. I just don't exactly know why. I want to be 100% with it before I improve it.

Each aiNode has a transformation attached to it. When I multiply all these transformations according to the node hierarchy I get a global transformation (toRoot). Each aiNode also has a number of submeshes attached to it. Each vertex these submeshes, before I multiply it by the global transformation (in the vertex shader, named ModelMatrix), is in local space. That might be the reason I get confused, because basically a vertex' local space isn't its rest pose position (at least in assimp). Does that make any sense?

BUT assuming it is in local space. Tell me if I got it correctly: Say I have two joints (bones are actually joints, right?) J1, J2, and J2 is a child of J1, and a vertex affected by both. I rotate each joint 10 degrees upwards. Now calculating the rotation of the vertex for J2 for example, I want the vertex to be rotated 10 degrees assuming J2 is the center, and that's why I multiply it by the inverse matrix of J2, before applying the rotation matrix of J2, right? Then continuing with J1 I do the same.

Untitled.png.c6fc62e4e3d56a8927cd6ae0c7ad239c.png

One little thing that doesn't get right for me is that J2 rotation matrix is actually about 20 degrees because I went down the hierarchy and multiplied J2.matrix = J1.matrix * J2.matrix beforehand. How does it work with the approach I said above?

Advertisement

http://ogldev.atspace.co.uk/www/tutorial38/tutorial38.html

I've not used assimp but on glancing on this page it seems to suggest that the 'local space' IS the rest pose...

Quote

The reason why we need this matrix is because the vertices are stored in the usual local space. This means that even without skeletal animation support our existing code base can load the model and render it correctly.

The diagram on the linked web page doesn't indicate that vertices are associated with a bone, but with the mesh as a whole, which is what I would expect. I'm guessing they mean that if you draw it without transform you should see the rest pose (but the phrasing is rather vague).

 

36 minutes ago, Pilpel said:

That might be the reason I get confused, because basically a vertex' local space isn't its rest pose position (at least in assimp).

Have you tried drawing it without any transform and taking a screenshot for us? If you could do that it would help shed some light.

Overall if it is not this, maybe someone who uses assimp can help. Often with this kind of matrix maths there may be multiple ways of achieving the same result and the workflow may be different.

1 hour ago, lawnjelly said:

Have you tried drawing it without any transform and taking a screenshot for us? If you could do that it would help shed some light.

Yes. Consider the next model made in Blender.

example.png.cf8fbfa8bb08086d8ef76d24284eb4e7.png

This is how assimp imports it:

import.png.a66116dd44a497cc4e25d5438f69e718.png

Now if I render the scene without setting ModelMatrix to each node's toRoot matrix (setting it to the identity) both submeshes are drawn at (0,0,0).

draw.png.53cb0cbda378a4f6c521606aca6b05a9.png

This was a static mesh but I'm pretty sure it works the same for animated meshes.

Can you address the last thing I said in my last post?

Quote

One little thing that doesn't get right for me is that J2 rotation matrix is actually about 20 degrees because I went down the hierarchy and multiplied J2.matrix = J1.matrix * J2.matrix beforehand. How does it work with the approach I said above?

 

5 minutes ago, Pilpel said:

This was a static mesh but I'm pretty sure it works the same for animated meshes.

Static meshes are unlikely to work in the same way as a skinned mesh. Try with a skinned mesh, to see whether it is in the rest pose when you do not apply transformation.

7 minutes ago, Pilpel said:

Can you address the last thing I said in my last post?

 

3 hours ago, Pilpel said:

BUT assuming it is in local space. Tell me if I got it correctly: Say I have two joints (bones are actually joints, right?) J1, J2, and J2 is a child of J1, and a vertex affected by both. I rotate each joint 10 degrees upwards. Now calculating the rotation of the vertex for J2 for example, I want the vertex to be rotated 10 degrees assuming J2 is the center, and that's why I multiply it by the inverse matrix of J2, before applying the rotation matrix of J2, right? Then continuing with J1 I do the same.

Assuming it is local space at the moment means nothing to me, vertices are defined either in bone space, or relative to the origin of the whole skeleton / mesh (rest pose space, which I think assimp is referring to as local space, as the local space of the node), I'm not sure what else local space could be referring to.

The way that you can rotate one joint by 10 degrees and the child joint moves by this and its own 10 degrees is hierarchical scene structure or scene graph. You should understand this completely before attempting skinning, skinning builds on this.

All skeletal animation / blending depends on an understanding of how these hierarchies work. In short you have a series of nodes (could be bones, empties, anything) in a tree structure, and each has a local transform. When animating you would animate the local transform, then by concatenation you recurse from the root node down through the tree and calculate the world transform by multiplying the parent world transform and the child local transform (this is forward kinematics).

Calculating the bone transforms and bones animation is almost a separate topic from skinning, you can animate objects without using skinning for instance, although they are often used for animals to get movement of the skin to match the underlying bone structure.

The reason the inverse rest pose is used is to get the vertex into the bone space of the bone in question, so it can be rotated using the correct origin.

It is quite difficult to explain in words, you really need to follow some more tutorials and pics and videos, and eventually you will 'get it'. I am getting the impression you have followed some tutorials but not understood completely what was going on under the hood. Here is wikipedia:

https://en.wikipedia.org/wiki/Skeletal_animation

Quote

For a polygonal mesh, each vertex can have a blend weight for each bone. To calculate the final position of the vertex, a transformation matrix is created for each bone which, when applied to the vertex, first puts the vertex in bone space then puts it back into mesh space. After applying a matrix to the vertex, it is scaled by its corresponding weight.

And don't feel bad, my first attempt at getting skinning working was 20 years ago, I remember very well it went horribly wrong and took me another couple of years before I had anything properly working!! (Not so many good tutorials and software back then!) :)

The code attached was 2 years old. Will sure watch the video to refresh my memory. Thanks a lot!!

This topic is closed to new replies.

Advertisement