Advertisement

Combining translation & rotations and per-frame rotations

Started by December 03, 2023 09:31 PM
2 comments, last by _Silence_ 11 months ago

I've been working on implementing animation using BVH files and I think I finally understand how to get the relevant data.

The part I am most confused with is how I should now apply and combine this translation and rotation data to have the animation work.

The localTranslation and localRotation matrices store essentially the translation and rotation to get the model in the appropriate pose so that when per-frame rotations are applied from bone_rotations, the animation will play as its a matter of just rotating the bones. I am confused on how you are supposed to combine these matrices.

My main confusion is about the parentMatrix. Some sources suggest it transforms the joint to bone-space, while others say it transforms to the global coordinate system. When I apply it to jointPos as: jointPos = parentMatrix * jointPos, my skeleton renders correctly in a hierarchical manner. Without this multiplication, it doesn’t. I’ve attached images to illustrate this and code showing how I am rendering this pose and how im storing the translation and rotation data which I then need to apply. Can anyone clarify this for me?

r/GraphicsProgramming - Combining translation & rotations and per-frame rotations
Applying the parentMatrix
r/GraphicsProgramming - Combining translation & rotations and per-frame rotations
Not applying parentMatrix
void Skeleton::drawJoint(Mat& viewMatrix, Mat4 parentMatrix, Joint* joint, float scale, int currentFrame)
{ 

    vec3 offset_from_parent = vec3(
    	joint->jointOffsetX,
    	joint->jointOffsetY,
    	joint->jointOffsetZ
    );
    
    auto jointPos = vec3(0.0f, 0.0f, 0.0f);
        
        // This is data to get the skeleton in the right pose
    Mat localTranslation = Mat::Identity();
    Mat localRotation = Mat::Identity();
    for(int i = 0; i < joint->jointChannel.size(); i++)
    {
    	const std::string& channel = joint->jointChannel[i];
    	float value = frameData[frame][BVH_CHANNEL[channel]];
    	if(channel == "Xposition")
    	{
    		localTranslation = localTranslation * Mat::Translate(vec3(value, 0.0f, 0.0f));
    	}
    
    	if(channel == "Yposition")
    	{
    		localTranslation = localTranslation * Mat::Translate(vec3(0.0f, value, 0.0f));
    	}
    
    	if(channel == "Zposition")
    	{
    		localTranslation = localTranslation * Mat::Translate(vec3(0.0f, 0.0f, value));
    	}
    
    	if(channel == "Xrotation")
    	{
    		localRotation = localRotation * Mat::RotateX(value);
    	}
    
    	if(channel == "Yrotation")
    	{
    		localRotation = localRotation * Mat::RotateY(value);
    	}
    
    	if(channel == "Zrotation")
    	{
    		localRotation = localRotation * Mat::RotateZ(value);
    	}
    }
    
    auto localTransformation = localTranslation * localRotation;
        // is parent for global transformation?
    auto globalTransformation = parentMatrix * localTransformation;
    
    // multiply the parent matrix to get offset in parent coordinate system
    jointPos = offset_from_parent;
    vec3 localPos = localTransformation * jointPos;
        
    // this will give rotation per frame to apply which will "play" the anim.
    auto boneRot = bone_rotations[frame][joint->id];
    // BVH stores rotation as Z,Y,X so, boneRot.x would be rotation for z
    auto z = boneRot.x;
    auto y = boneRot.y;
    auto x = boneRot.z;
    //z,y,x multiply like: x,y,Mat::RotateX(x) * Mat::RotateZ(z)
    auto localMatrix = Mat::Translate(jointPos); 
    auto localBoneRot = (Mat::RotateX(z) * Mat::RotateY(y) * Mat::RotateZ(x));
    auto matrix = localMatrix;
    
    for(auto& child : joint->Children)
    {
        vec3 end = matrix * vec3(child.jointOffsetX, child.jointOffsetY, child.jointOffsetZ);
    drawCylinder(viewMatrix, jointPos, end);
        // Recursively render the child joint
        drawJoint(viewMatrix, matrix, &child, scale, currentFrame);
    }

}

snoken said:
My main confusion is about the parentMatrix. Some sources suggest it transforms the joint to bone-space, while others say it transforms to the global coordinate system.

Try out any options to guess the used convention. That's often easier than understanding given specifications, because such specs might be hard to formulate and read.
And because we can't be sure either, it's hard to help.

But there are some issues with your code:

You do everything twice.

You have a offset_from_parent, assuming it gives the offset of the child bone in the parent frame. With regular skeletons that's constant and will never change, thus there is no need for animated position data at all. We only need animation data for rotations.
But you also have position animation data. And you somehow use it too. This ofc. adds to the confusion.

You have rotation animation data, and you use it to make a localRotation matrix. Notice the Euler angles order comes from the order of the data. If Xrotation comes first, it will be the first angle. If it comes last, x rotation happens after the other two axis. So you basically have a random order which likely is not right or might work only for some files due to random luck.
But you have this a second time as well here: localBoneRot = (Mat::RotateX(z) * Mat::RotateY(y) * Mat::RotateZ(x)), coming from here: bone_rotations[frame].
At this point i can't make a guess what either should mean, and i do not wonder you're confused.

However, let's first clarify the Euler angles order thing with some example.

// BVH stores rotation as Z,Y,X so, boneRot.x would be rotation for z

Where do you get this from? If it's specified in the file, please post the section. As said before, i think you got it wrong, and they mean that they use ZYX Euler angles order.
If so, you do not have to swap dimensions, you have to swap the order of rotations. Code would be:

auto boneRot = bone_rotations[frame][joint->id];

// BVH stores rotation as Z,Y,X; assuming it refers to Euler angles order we need to build rotation in this order:

Mat4 localBoneRot = Mat::RotateZ(boneRot.z); // z first
localBoneRot *= Mat::RotateY(boneRot.y);
localBoneRot *= Mat::RotateX(boneRot.x); // x last


// but depending on your math libs multiplication order convention, you might eventually need to do this instead:

// Mat4 localBoneRot = Mat::RotateZ(boneRot.z); // z first
// localBoneRot = Mat::RotateY(boneRot.y) * localBoneRot; // <- swapped multiplication order
// localBoneRot = Mat::RotateX(boneRot.x) * localBoneRot; // <- swapped multiplication order
// ... one reason why it's hard to help from a distance!

I hope this makes it clear. Order matters. Euler angles describe a combination of 3 rotations. So we need to know the angles AND the order.
Animation file formats usually do specify Euler angle order, and usually they do this per individual bode, because artists may prefer XYZ for a shoulder, but YXZ for a hip, and ZXY for a wrist.
Artists may setup their skeletons with multiple orders so it's easier to work around gimbal lock while animating manually.

To deal with this mess, we usually preprocess assets. That's an offline tool which imports 3D models from various file formats, but brings everything to the same convention our engine uses.
At this point we will ditch Euler angles all together and replace them with quaternions. After that we can mix multiple animations easily, and there is no more need to combine 3 rotations and no more confusion about Euler angles order.
Euler angles are useful for a human interface, but they are a terrible representation of rotations in a technical sense. So we want to get rid of them as soon as no more human edits are needed.
That just said to give some background. It's fine you try to display the data from file directly, but on the long run you'll need to do things a bit differently, defining your own data structures / conventions and converting file data to that.

I can also give some background on related data structures.
If our goal is just character animation playback, our naive data structure might look like this:

struct Bone
{
	Vec3 jointPosInParentSpace; // constant, never changes
	Quaternion currentFrameRotationFromAnimationData; // changes each frame
	Mat4 currentTransformInWorldSpace; // that's what we fanially want to calcualte from the above data over the skeleton hierarchy
};

That's quite simple, and we would need animated positions only for the root bone eventually. All other joint positions are calculated from current rotations.

But it becomes more complicated if we also want a physical representation of our character. Because now we don't have ‘bones’ anymore. We have rigid bodies (e.g. capsules for upper and lower arm) and joints (e.g. hinge for elbow and cone twist for wrist):

struct RigidBody
{
	Mat4 currentTransformInWorldspace; // changes each frame
	Vec3 linearVelocity, angularVelocity; // changes each frame
	float mass; // remains cosntant
	Vec3 localInertia; // remains constant
	// ... some other things, e.g. definign the collision shape 
};

struct Joint
{
	RigidBody *body0, *body1; // remains constant
	Mat4 jointSpaceOffsetFromBody0, jointSpaceOffsetFromBody1; // remains constant; used to calculate the current position of the elbow joint from both upper and lower arm bodies, and the constraint solver will try to bring those two poitns together
	// ... some other things like joint limits
};

Notice how physics representation raises the complexity of our data structures.
And notice the position of our upper arm no longer is given by some parent bones joint offset. Instead our position is now at the center of the upper arm, so between the shoulder and elbow joints.
Although both our animation / physics data structures represent the same character, there is a fundamental difference on how we lay out the data. In animation a bone describes both the body and its joint position, in physics we use two individual data structures.

I guess the bone data structure is enough for you to know when importing animation files, but maybe not and knowing this might help.

Advertisement

snoken said:
I am confused on how you are supposed to combine these matrices

The normal standard order for transformations is T.R.S. I'm unsure if BVH follows this however.

For BVH and the rotations order, it seems to be Y,X,Z: https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html

This topic is closed to new replies.

Advertisement