Ok my idea is to rotate my avatar after it collides with an object in the game and I also need the camera to rotate 180 degrees. I at first thought just use a rotation matrix around the Y axis and walla... Not working I think it has to do with not being local to the model space?


I am assuming I can just rotate the position of the camera and the objects position?

If I understand you correctly, then you want an orbiting of the camera around the colliders position. With P and O meaning the camera's current position and orientation, resp., and C meaning the colliders position, orbiting with a rotation R means to compute
C * R * C[sup]-1[/sup] * ( P * O )
here written down using column vectors. Because the camera is a bit apart from the collider, C and P are not the same and hence
C[sup]-1[/sup] * P != I
meaning that R will change both the position as well as the orientation of the camera.

If you throw the same formula to the object directly, then P is actually the same as C, so that (this time using O for the orientation of the model)
C * R * C[sup]-1[/sup] * ( C * O ) = C * R * O
will perform a model local rotation.

Yeah, I am not 100% following this, but are these all vectors or positions? I am not doing any vector math on this.

Here is the camera code and my objects update()


//camera
void Update(void)
{
cml::matrix44f_c rot, trans, rotWorldY;
trans.identity();
rot.identity();
rotWorldY.identity();

cml::matrix_rotation_world_axis(rotWorldY, 1, cml::rad(180.0f));

cml::matrix_translation(trans, 0.0f, 0.0f, -radius);
cml::matrix_rotation_world_axis(rot, 0, cml::rad(xRot));
transformAvatar = trans * rot;

cml::matrix_translation(trans, -position[0], 0.0f, -position[2]);
cml::matrix_rotation_world_axis(rot, 1, cml::rad(yRot));
rotY = rot;
if(collision)
{
//transformAvatar *=rotWorldY;
rotY *=rotWorldY;
transform = transformAvatar * rot * rotWorldY * trans;
collision = false;
position = cml::transform_point(rotWorldY, position);
// position[0] += -2.0f;
// position[2] += -2.0f;
}
else
transform = transformAvatar * rot * trans;
}
inline void Move(float t)
{
float yRotRad = yRot / cml::constants<float>::deg_per_rad();
float xRotRad = xRot / cml::constants<float>::deg_per_rad();
position[0] += sinf(yRotRad) * t;
position[2] -= cosf(yRotRad) * t;
position[1] -= sinf(xRotRad) * t;
}



//objects
void Player::Update(void)
{
NX::App* app = NX::App::Get();
position = app->GetCamera()->GetPosition();
transform = app->GetCamera()->GetAvatarPosition();
if(collision)
{
cml::matrix44f_c rot;
rot.identity();
cml::matrix_rotation_world_axis(rot, 1, cml::rad(180.0f));
transform *=rot;
collision = false;
}
UpdateAABB();
}


Yeah, I am not 100% following this, but are these all vectors or positions? I am not doing any vector math on this.

All bold written capital letters in my post above are transformation matrices. E.g. O is the rotational matrix that gives the camera its orientation in the world, and P is the translational matrix that gives its position in the world. Because we use so-called homogeneous vector (those with mysterious 1 in the 4-th dimension) we are enabled to concatenate all usual transformation by multiplying the matrices together. That is what I've done in my post above.

When looking at the expression
C * R * C[sup]-1[/sup] * P * O
and remembering that I've used column vectors for explanation, the things that happen to the camera are these:
1. Rotate the camera so that it gets its world orientation: O
2. Translate the camera so that it gets its world position *after* doing the rotation: P * O
(Notice that the position doesn't influence the orientation; you can see this when actually multiplying out the matrices on paper.)
3. Translate the camera by the inverse (hence the [sup]-1[/sup] in C[sup]-1[/sup]) of the colliders position C, so C[sup]-1[/sup] * P * O
Of course, especially the inverse of a translation is in fact a translation by the negative amount:
C(dx,dy,dz)[sup]-1[/sup] = C(-dx,-dy,-dz)
Hence we make a space where the collider is placed in the origin. The sense of this step is that 0 (vector null) is ever part of a rotation axis, and we want to rotate around the collider's origin, don't we?
4. Then we do the "orbiting", in your special case a rotation by 180° around the y axis, denoted by R, so: R * C[sup]-1[/sup] * P * O
5. At least undo step 3 so that the world isn't shifted any more: C * R * C[sup]-1[/sup] * P * O

I will take a look at your explanation closer after a bit, but is the code I posted correct? and will your example work with what I am doing? I don't want to recode my code to do something else....