In my initial code I had player movement applied directly as offsets to the position and orientation data.

My idea was to try and change that, so that player movement and rotation was instead applied once to the player's velocity or angular velocity vectors.

With simple translation this is easy. However, when the player rotates, the translation vector needs to change direction. This means subtracting the old vector and adding the new one. To complicate things further, the same is true of the 'up' vector if the player is pitching downwards.

So before each physics update the old vectors are subtracted, a new orientation matrix is calculated for the next frame, and the velocity vectors are re-applied in the proper directions.

This worked for either translation or rotation alone, but when moving and rotating simultaneously I ended up with residual velocity (around 0.02-0.07 units/sec, compared to a movement speed of around 1.6-8.0 units/sec). It's small enough that it seems like it could possibly be a rounding error but also not so small that there could be some small error in my algorithm, or else the method described above is just not possible.

I wanted just one set of velocity and angular velocity vectors, so that during the physics update step all collision could be handled in one step. Otherwise I will have to make a second collision step if I simulate player movement separately. This seems undesirable since my player is just an instance of a mobile game entity which I will be adding later, so all these objects would add more overhead to the movement step.

Suggestions?

This sounds like a 3D scenario yes? The answer is not simple in that case. You are going about this correctly for position...rotation is another story.

Angular velocity is generally represented with a vector called omega. The direction of omega represents the axis of rotation and the magnitude represents the velocity of the rotation. You can think of omega as the time derivative of your rotation matrix R; however unlike position you can't simply multiply omega by your time step and add it to R. It's a bit trickier than that but it's not terrible. The formula for what you want to do is here:

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=1&ved=0CC0QFjAA&url=http%3A%2F%2Fwww.cs.cmu.edu%2F~baraff%2Fsigcourse%2Fnotesd1.pdf&ei=YgsHUvzCBOWh2gW13oHIAw&usg=AFQjCNHGXY6ZN2RYXF7f28loeidm1rOxZQ&sig2=8P7y8zCkm6aBEgpeuIS0vw&bvm=bv.50500085,d.aWc

at the bottom of page 8. This approach is going to lead you to another problem...how do you change omega to get rotations that correspond to the player's input? This requires applying forces, computing the torque, and then integrating the torque with the help of an inertia tensor to compute omega. Again, not hard but all the texts that explain how to do this require some familiarity with calculus and some patience.

[quote name='nonoptimalrobotND' timestamp='1376194105' post='5084849']
however unlike position you can't simply multiply omega by your time step and add it to R[size=NaN][background=#d6e3fe ! important]F[/background]

I do use the rotation function from the glm library:

//apply angular velocity to orientation
const float langle = glm::length( lrmotion.mAngularVel );

if( langle != 0.f )
{
    lrorient.mOrient = glm::mat3( glm::rotate(
        glm::mat4(
            glm::vec4( lrorient.mOrient[0], 0.f ),
            glm::vec4( lrorient.mOrient[1], 0.f ),
            glm::vec4( lrorient.mOrient[2], 0.f ),
            glm::vec4( 0.f, 0.f, 0.f, 1.f ) ),
        langle,
        lrmotion.mAngularVel ) );
}

I see, I misunderstood your question. While the method you are using to apply an angular velocity to your rotation isn't exactly "normal" it does get the job done. It doesn't extend well to doing things the "real" way with a physically derived angular velocity but you don't seem to be considered about that.

The 1st, 2nd and 3rd row of your orientation matrix are actually the local x, y and z axis of the player's local coordinate system specified in world space.

mtxRotation = | ux vx nz |
              | uy vy ny |
              | uz vz nz |

x-axis = [ux, ux, uz]
y-axis = [vx, vy, vz]
z-axis = [nx, ny, nz]

To adjust the players translation in a consistent way regardless of the current orientation:

vPosition += [ux, uy, uz] * vVelocity.x * fDeltaTime
vPosition += [vx, vy, vz] * vVelocity.y * fDeltaTime
vPosition += [nx, ny, nz] * vVelocity.z * fDeltaTime

Note that this is the same as:

vPosition += mtxRotation * vVelocity * fDeltaTime

In this scenario your velocity is always defined local to the players orientation. The x-component always corresponds to left-to-right motion, the y-component always corresponds to vertical motion and the z-component always corresponds to forward-to-back motion.

My problem seems to be with rotation (adding vectors).

I found the proper formula here for axis-angle representation:

http://www.physicsforums.com/showthread.php?t=542904

It uses sin and cos functions which I think are computationally expensive? Are there version of these functions that use dot and cross products instead?

My problem seems to be with rotation (adding vectors).

I found the proper formula here for axis-angle representation:

http://www.physicsforums.com/showthread.php?t=542904

It uses sin and cos functions which I think are computationally expensive? Are there version of these functions that use dot and cross products instead?

Are you talking about the formulas for composition of rotations? If you express rotations by quaternions or matrices, all you need to do is multiply them.

Also, how many times per frame do you expect to call these trigonometric functions? Even if it's in the hundreds you probably shouldn't worry about it from the point of view of performance.

As many times as possible. I see I can convert to a quaternion representation but that will also involve sin and cos functions. It might be unavoidable, I just haven't used any trig functions directly in my code, probably because I mostly rely on the glm library and actually it looks like it has quaternion facilities available so I should probably just revert to that approach.

Question about composing/decomposing rotations.

If you have two rotations Q1 and Q2, then the result of rotating first by Q1 then Q1 is Q':

Q' = Q2Q1

If you have Q' how do you get back to Q1? Is it rotating by Q' then rotating by the inverse of Q2?

Q1 = inverse(Q2)Q'

Similarly, could you start with Q' and remove the Q1 rotation, leaving the Q2 rotation? I'm trying to find a clear set of operations and identities that I can use algebraically for rotations.

You can pre- and post-multiply by matrices on both sides without changing the validity of a matrix equation.

So to obtain Q2 from

Q' = Q2Q1

you would post-multiply by inverse(Q1) on both sides

Q' * Q1^-1 = Q2 * Q1 * Q1^-1

Q' * Q1^-1 = Q2 * I = Q2

EDIT: So Q2 = Q' * inverse(Q1)

You can do the 2 multiplies on the right hand side in any order (without changing which matrices are on the left or right) since matrix multiplication is associative:

A * B * C = (A * B) * C = A * (B * C)

but not commutative (i.e. AB != BA in general).

EDIT2: Similarly for quaternions, you can pre- and post-multiply equations by a quaternion, and they are associative but not commutative.

If you are trying to find identities and operations for rotations/matrices in general then the following may be of use as well

(AB)^-1= B^-1A^-1

If a matrix is orthonormal (all rows are perpendicular [orthogonal] to each other and normalised, as are the columns)

A^T = A^-1

where A^T = transpose of A. 3x3 Rotation matrices are always orthonormal (as are 4x4 rotation matrices with no translation component).

Regardless of whether matrices are orthonormal (i.e. this is always true)

(AB)^T = B^TA^T