So 3DS max appears to be left handed and has the z be pointing up, and y pointing forward.

My engine uses a right handed coordinate system where y is up.

I'm trying to write a max script that exports the names of selected objects and their transform, and also puts the transform into my space.

At first I was trying to just transform the matrix by rotating it 90 degrees about the x axis and scaling it negatively in the z.

In the end, the rotation portion looks like

1 0 0
0 0 1
0 1 0

I want it to basically be an identity matrix with just positions remapped if the node isn't even rotated.

I remember on Android there was a function to switch coordinate spaces in a matrix as well.

http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/android/4.0.1_r1/android/hardware/SensorManager.java#SensorManager.remapCoordinateSystem%28float%5B%5D%2Cint%2Cint%2Cfloat%5B%5D%29

They seem to have some crazy bit-flipping going on that I'm having a hard time understanding.

I basically want my engine to load in the scene exactly as it appears in the viewer, only making all positions and scale be in my coordinate system. If an object isn't rotated I want the diagonal to just be identity.

This also means that in 3DS max I want an object at negative y to be remapped to positive Z, and an object at positive Z to be remapped to positive y.

Can't you just export your models in the proper coordinate system? I know for a fact that OBJ exporting lets you do that (if my memory isn't failing me). Or you could also change the 3DS viewport settings to use whichever coordinate space you want, thereby sidestepping the whole issue without a single line of code.

I've tried using things like swapping the up axis when exporting to dae or fbx and those made no difference sometimes.

Right now though I'm not exporting geometry. I'm writing a script that exports the names of objects and their transforms and bounding boxes. I currently don't have a level editor so this is the easiest way for me to make environments.

I have various hacky ideas for how to do it that I've used back in the day but I thought I'd learn how to do it right once and for all, if there's a way.

I figured it out.

I remember seeing another post that said to do this, but wasn't flipping the z value. I was making the mistake of scaling z negatively instead of just negating the position component.

Now I'm doing this:

This is my swapping transform matrix, which pretty much rotates by 90 degrees on the x axis:

1 0 0 0
0 0 1 0
0 1 0 0
0 0 0 1

yzSwap * transform * yzSwap puts it into a left handed coord system with y being up.

Then in the final transform I take the translation z portion, and negate it to put it into my right handed coord system.

transform[3][2] = -transform[3][2]