assuming a left hand coordinate system:
when the game starts, the ship is at rotation angles 0,0,0, so forward points in the direction of the positive z axis, right points in the direction of the positive x axis, and up points in the direction of positive y axis.
now we apply a turn. we'll hit left arrow once, which turns us 5 degrees left. so we want to do an axis-angle rotation around the up vector. so we send our up vector as the axis, -5 degrees as the amount (remember we're turning left, and right is positive), and the end of the forward vector as the point to rotate. call the axis-angle (rotate point about arbitrary axis) routine using those parameters. this gives us a new point, whichis the end of our new forward vector. now do the same for the right vector: up is the axis, -5 degrees is the angle, and the end of the right vector is the point. this gives us our new right vector. since we're rotating around the up vector, it doesn't move, so it remains unchanged. the result is you've applied a discrete finite rotation of -5 degrees around the local y axis, and you now have your new forward, up and right vectors, representing the new orientation of the ship. so far so good. only problem is, due to floating point error, the vectors are not quite orthogonal any more, and they are no longer exactly of unit length. not enough to make a big difference yet, but the error accumulates over time, and things get whacked beyond use within minutes. so sooner or later you're going to have to re-ortho-normalize.
now its time to draw. stuff your vectors into a mat (put the components in the correct rows/columns). this will be your rotation matrix. create your scale matrix, cat on your rotation matrix, cat on your translation, and draw.
ok, time to turn again. we have a ship which is currently pointing 5 degrees left of looking down the z axis. its orientation is stored in our vectors, and also happens to be in the rotation matrix we used for drawing. don't really need it in both places, this is why i use mats. i just keep the rotation matrix around all the time and change the vectors in it as needed, so i'm storing the vectors in the matrix. lookup which rows/columns of a matrix handle rotation - all this will make much more sense.
so we're turning.... this time we pitch up 10 degrees and roll left 10 degrees. our rotation order is always xr, yr, zr, (or whatever you choose) so we pitch (xr), then roll (zr).
so call axis-angle: axis=right, angle=-10 (pitch down is positive), point = end of the forward vector. this gives us our new forward vector.
and axis=right, angle=-10 (pitch down is positive), point = end of the up vector. this gives us our new up vector.
now roll:
call axis-angle again: axis=new forward (NOT old forward, we've already pitched!), angle = 10 degrees (positive roll is counter clockwise or left - remember your left hand rule). point = end of the right vector. this gives us our new right vector.
and axis=new forward, angle=10, point = end of NEW up (it changed when we pitched) gives us our "new new" up <g>.
again, float errors will occur requiring periodic re-ortho-normalization.
and now the general case:
roll by amount zr, pitch by amount xr, and yaw by amount yr:
assume our rotation order is xr,yr,zr.
so we pitch first:
axis = right, angle = xr, point = end of forward vector => new forward vector
axis = right, angle = xr, point = end of up vector => new up vector
then yaw:
axis=new up, angle = yr, point = end of new forward vector => new new forward vector
axis=new up, angle = yr, point = end of right vector => new right vector
now roll:
axis=new new forward, angle = zr, point = end of new right vector => new new right vector
axis=new new forward, angle = zr, point = end of new up vector => new new up vector
so your final orientation is given by the new new forward vector, the new new right vector, and the new new up vector.
but again, you're skewed and not unit length. so now we re-ortho-normalize. do the cross product algo i described, followed by normalizing the vectors.
note that the orientation is stored in the vectors, and they always point in the correct directions in relation to the ship, and are not recalculated every time.
it IS possible to do that though. once you have your new vectors after rotating the ship, you can used the forward vector to solve for the equivalent xr and yr euler angles (global rotations). roll is a bit trickier. i don't do this myself, so i'm not 100% on this, but once you have the global xr and yr eulers, you un-rotate ,up around the global y axis by the yr euler angle, , then un-rotate it by the xr euler angle. remember, you're un-rotating around global axes here, so your order is reversed, zr, then yr, then xr, and all your rotations are negated: -10 instead of 10, for example, and you use the global rotation formulas. once you do that roll should be the angle between the y axis and the projection of the up vector onto the xy plane. this gives you three global euler angles equivalent to the orientation of the vectors. from there you rebuild the matrices and vectors again next frame. all in all, not a trivial amount of work. now you see why i (and most folks) don't do it.
whenever you concatenate successive small rotations to an orientation matrix/quat/vectors, skewing occurs due to float error. double doesn't help - just delays the problem. converting to eulers doesn't really help since you're using skewed vectors to calculate the eulers. so re-ortho-normalization is unavoidable, and will very soon become a very good friend of yours <g>.
quick answer to you previous post: store in vectors, just keep rotating them, re-othero-normalize from time to time. not sure what you're doing in your code, but it doesn't look like it does the same thing as the algos i'm talking about. the way you create your rotation matrix looks especially suspicious. compare your code to the above algo and you should soon find where you'e going astray.
and soon you too will have "the amazing flying axis brothers" (up, right , and forward vectors as placeholder graphics for testing local rotations) swooping, diving, and barrel rolling across your screen. <g>
so, now hopefully all is clear, and in two weeks we get to see your demo of pixel perfect local rotations, right? <g> (just kidding).
actually, odds are it won't even take you that long to get it working, a few days maybe.