I am working on merging animation between different skeletons. It works by comparing the current keyframe to the base pose, and the adding that relative difference to the original skeletons base bose. It is working pretty well, but there is an issue when the size are different.

If you look in the Gif, you can see the feet of the right skeleton are sliding around. I am pretty sure this is because the relative rotation is causing a different "world" position of the next bone. Any ideas on how can I correct for this?

i think your first problem is Tom Cruise wouldn't be too happy that these 2 mission dudes are so closed to each other, staring in the same direction, shaking the same cocktail and breaking covid safety rules -lol-

anyway, you need to retarget the translation components of them bones;

read the section “How Does Retargeting Work?” here: https://docs.unrealengine.com/en-US/AnimatingObjects/SkeletalMeshAnimation/AnimationRetargeting/index.html

and it should give u an idea of what to do;

unreal's code is also up for grabs, so have a look in there too;

all the best ?

I was really confused for a bit there!

I did have a look at the unreal code some time ago, but to be honest I could not really figure out how they are doing it. I have never looked at the unreal code before so I got a bit lost.

I am trying to visualize what is actually going on/Rubber ducking.

Lets say I have source skeleton B and target skeleton A. Green is the floor, and the red circle is the pelvis (root bone). The values are rough estimates to get my mind working.

The length of A0 to A1 and A1 to A2 is 2. For the B this is 1. For the foot to rest on the floor with the new pelvis position, the two angles need to be updated. The difference in rotation is -45 for the knee and +45 for the foot. The total rotation value of the system is the same, 405. The relation between the bones are 2:1. Can I use that ratio?

Bones don't have length. They are biped pose defintion matricies that keep quternion/rotation information only afterwards.

It is those biped pose matricies that define their individual position, thus some length infromation etc.

If you target a different mesh, you target a different biped pose, that has all animations playable.

True, there are not any direct “bone length” but one can easily be calculated.

If you target a different mesh, you target a different biped pose, that has all animations playable

What do you mean by that?

Some more thoughts:

B1 Angular distance relative to B0 = -1
A1 Angular distance relative to A0 = -1.207
Ratio = 0.828

B2 Angular distance relative to B1 = 0.5
A2 Angular distance relative to A1 = 0.414
Ratio = 1.2

original angle B1 = 315
new angle A1 = 315*0.828 = 260.82

original angle B2 = 90
new angle A2 = 90*1.2 = 108

I think that might make sense? Looks like I actually have to go and buy a protractor so I can work this out on paper.

h3ro said:
Can I use that ratio?

yes you must ?

h3ro said:
I think that might make sense?

yes that is correct; yes it is a scaling up or down of your existing bone translations, going from a small character to large you multiply the or then divide the ratio for the opposite retargeting case)

//pseudo

boneA1.pos = boneB1.pos * ratio (when retargeting to a larger skel of the same skel)
boneA1.pos = boneB1.pos / ratio (when retargeting to a smaller skel of the same skel)

U can do all sorts of other tech but that's the main gist

well done ?

Note1: u can store your ratios with your skeleton data so u don't need to recalc them all the time

Note2: if you are retargeting from a different skeleton to another skeleton then you must use a RIG (remapping your bone from one to another by name, it's a manual process, so having a UI that offers this, would help)

@ddlox

Alright, I will give that a go.

The skeletons are different, but I have a UI for it. When you say RIG, is that just a UI for saying Bone 0 related to Bone 3 in Skeleton B, or is it something else?

I am not sure I understand you pesudo code. Most of the time the animation is just a rotation. Is the Pos attribute in your example the full matrix transform of the bone? Or should I transform the bone into world position, then do the scaling before creating the local matrix again?

When calculating bone length, its correct to compute bone → bone.Parent?

So the end result should look something like this?

var relativeTransform = ComputeRelativeTransform(reMappedBoneIndex, frameIndex, otherAnimation, otherSkeleton);
var skeleteonTransform = GetSkeletonTransform(boneIndex, currentSkeleton);

var currentBoneLenth = ComputeBoneLength(GetBone(boneIndex), GetParentBone(boneIndex), currentSkeleton);
var otherBoneLenth = ComputeBoneLength(GetBone(reMappedBoneIndex), GetParentBone(reMappedBoneIndex), otherSkeleton);

var ratio = currentBoneLenth / otherBoneLenth;

var finalBoneTransform = skeleteonTransform * (relativeTransform * ratio) // Devide if smaller.

It might be useful to work with positions as well.

E.g. if you know target positions of A0 and A2, A1 end's up on the blue circle, and all angles can be calculated easily.

I assume it would make sense for you to use such method mainly for feet (or other resting contact / strict targets).
Ofc. it's not easy if size of characters differs, but it can be used to keep a foot in place.

h3ro said:
When you say RIG, is that just a UI for saying Bone 0 related to Bone 3 in Skeleton B

yes, but u may find that rigging some bones this way alone will not suffice (explained at the end of this post)

h3ro said:
Is the Pos attribute in your example the full matrix transform of the bone?

no, it's just its translation part, for example:

// pseudo

struct bone
{
 quat rotation; 	// relative to parent bone
 float3 scale;		// relative to parent bone
 float3 pos; 		// relative to parent bone <--- it's this fella 
};
// thus, the local space bone matrix relative to its parent would be (in right-to-left notation):
matrix4x4 boneM = to_matrix_4x4( (ratio * pos) * rotation * scale ) // normalise where necessary
or
matrix4x4 boneM = to_matrix_4x4( (pos / ratio) * rotation * scale ) // normalise where necessary

h3ro said:
Or should I transform the bone into world position, then do the scaling before creating the local matrix again?

you can do it in the bone's local space as in my example above ^^^^^^

When calculating bone length, its correct to compute bone → bone.Parent?

yes, why? because if they were projected to the same plane, you'd end up with linear angle-angle similarities

So the end result would look something like this?

yes, except the application of the ratio need only be done on the bone position, as u can see such as B = D / ratio, etc… and the angle [of rotation] can remain the same (if the skeleton is the same).

But if you're using different skeletons then yes you do want to play with the rotation angles as well to adjust them (if rigging alone doesn't suffice)

hope this clears it now;

have fun ?

Ok. I think I understand now, it will probably improve the animation a bit, but I dont understand how it addresses the real problem.

Today I do this for position:
Position = CurrentSkeletonBone.Pos + (OtherAnimBone.Pos - OtherSkeletonBone.Pos);
This makes more sense I guess
Position = OtherAnimBone.Pos * BoneRatio;

But, it does not address the problem of changing size I think. If a bone is longer in one skeleton, the world position will be different, as the angular distance traveled by that rotation is larger then what it originally was. Because of this, following bones will have the wrong angle in correlation to that. Which again is what is causing the sliding in gif in the first post.

So for the leg in my drawn example to keep its foot position on the ground, I need to actually modify the rotation of the bones.

Or am I completely missing the point here?

Edit: I just updated my code to use the bone ratio as a scale, and its the same result as using the old “relative position” approach. The feet are still floating