I'm trying to implement skeletal animation from a glTF2 file with a very simple rigged model.

https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/RiggedSimple

It has only two joints and three key frames.

I have a Skeleton with a list of Joints. Each Joint has a position, rotation, scale and a inverse bind matrix, which I load directly from the glTF2 file.

My Animation has a list of Tracks and each Track has a list of KeyFrames. Nothing special so far.

I use the second KeyFrame, no movement right now, so the mesh should be bend.

Now I have a AnimatedSkeleton which is a copy from the Skeleton and I apply the modifications from the KeyFrame to it.

I recalculate all transforms on the AnimatedSkeleton, multiply each Joint world transform with the corresponding inverse bind matrix and fill my JointMatrix array.

In RenderDoc I can see that my JointMatrix array gets filled, unfortunately the mesh is still in his binding pose.

Quick question: It is correct that I replace the values and not add them right?

So I do:

animatedSkeleton.joint.rotation = animation.keyFrame.rotation

and not

animatedSkeleton.joint.rotation += animation.keyFrame.rotation

Here is my HLSL vertex shader skinning code:

matrix GetSkinningMatrix(VSInput vin)
{
	matrix skin = Identity;

	#if defined(HAS_WEIGHT_SET1) && defined(HAS_JOINT_SET1)
    skin +=
        mul(SkinJointMatrix[int(vin.joint0.x)], vin.weight0.x) +
        mul(SkinJointMatrix[int(vin.joint0.y)], vin.weight0.y) +
        mul(SkinJointMatrix[int(vin.joint0.z)], vin.weight0.z) +
        mul(SkinJointMatrix[int(vin.joint0.w)], vin.weight0.w);
    #endif

	return skin;
}

float4 GetPosition(VSInput vin)
{
	float4 pos = float4(vin.position, 1.0);

#ifdef USE_SKINNING
	pos = mul(pos, GetSkinningMatrix(vin));
#endif

	return pos;
}

And my SkeletonJoint methods to recalculate the transform and fill the JointMatrix array.

void CSkeletonJoint::ApplyTransform(const xmath::Matrix& parentWorldTM)
{
  xmath::Matrix scaleMat = xmath::Matrix::CreateScale(m_scale);
  xmath::Matrix rotMat = xmath::Matrix::CreateFromQuaternion(m_rotation);
  xmath::Matrix translateMat = xmath::Matrix::CreateTranslation(m_translation);
  
  m_localTM = xmath::Matrix();
  m_localTM *= scaleMat;
  m_localTM *= rotMat;
  m_localTM *= translateMat;
  
  m_worldTM = parentWorldTM * m_localTM;
  
  for (CSkeletonJoint& childJoint : m_children)
  {
    childJoint.ApplyTransform(m_worldTM);
  }
}

void CSkeletonJoint::Apply(std::vector<xmath::Matrix>& jointMatrices)
{
  xmath::Matrix jointMatrix = m_inverseBindMatrix * m_worldTM;
  jointMatrices[m_index] = jointMatrix;
  
  for (CSkeletonJoint& childJoint : m_children)
  {
    childJoint.Apply(jointMatrices);
  }
}

I do not recalculate the inverse binding matrix, I use it from the glTF2 file since I need the unmodified version right?

I just don't see where I miscalculated the JointMatrix.

I didn't found a "Edit" button so I'm creating a new reply.

I changed my Skeleton and SkeletonJoint class so that Skeleton has only a list of Joints and each SkeletonJoint has a pointer to it's parent, but no children.

xmath::Matrix CSkeletonJoint::GetLocalMatrix()
{
  xmath::Matrix localTM = m_transformMatrix;
  localTM *= xmath::Matrix::CreateScale(m_scale);
  localTM *= xmath::Matrix::CreateFromQuaternion(m_rotation);
  localTM *= xmath::Matrix::CreateTranslation(m_translation);
  return localTM;
}

xmath::Matrix CSkeletonJoint::GetWorldMatrix()
{
  xmath::Matrix worldTM = this->GetLocalMatrix();
  CSkeletonJoint* pParent = m_pParent;
  while (pParent != nullptr)
  {
    worldTM = pParent->GetLocalMatrix() * worldTM;
    pParent = pParent->GetParent();
  }
  return worldTM;
}

I fill the JointMatrixArray in my Entity.

for (int32_t i = 0; i != pSkeleton->GetJoints().size(); i++)
{
  resource::CSkeletonJoint* pJoint = pSkeleton->GetJoints()[i];
  xmath::Matrix jointMatrix = pSkeleton->GetInverseMatrices()[i] * pJoint->GetWorldMatrix();
  jointMatrix *= m_pEntity->GetWorldTM().Invert();
  m_jointMatrices[i] = jointMatrix;
}

I use Matrix::Transpose before I send it to the shader and I already tried to change the matrix multiplication order.

If I rotate the first Joint(root) than the whole mesh rotates, but if I rotate the second Joint(child) it doesn't change at all.

I already looked at the glTF2 reference implementation(WebGL) and other skeletal animation implementaions and I still can't find my multiplication error.

It works now

Short explanation: Broken joint values in vertex buffer.

Long explanation: I checked multiple times if my joint or weight values from the glTF2 file were correct. I compared my values in RenderDoc with another program(which works) and everything seemed fine. But then I checked my values in Nvidia Nsight and it showed me broken values. I reworked my glTF2 importer and with the correct vertex buffer values my animation pose works.

I have no idea how people can work without RenderDoc/Nsight.