These matrices ( A & B ) both qualify as being orthogonal.

\[ \left [ \begin{matrix} 1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{matrix} \right ] \]

\[ \left [ \begin{matrix} -1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{matrix} \right ] \]

Furthermore, my code says their quaternion representation is actually the same ( both the identity quaternion). glm gives me { 1/sqrt(2), 0, 0, 0 } for B curiously.

Also I know that det(A) = 1 and det(B) = -1

So I guess that orthogonal matrices have some sort of "sign" property to them, that gives the handedness of the basis? And how this work with quaternions? I suspect it might have some relation to the famous 'double cover' property of quaternions that people always mention. Basically the fact that Q and -Q correspond to the same rotation. I also wonder if this works for complex numbers as well as they're the 2d equivalent of quaternions.

Are there any other properties involving this 'sign'? I have a vague feeling it had something to do with the diagonal as well.

Maybe someone here has some deeper insights or facts to share about this

That second matrix is not orthogonal and its determinant is zero. Did you mean this one?

\[ \left [ \begin{matrix} -1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{matrix} \right ] \]

That second matrix is not orthogonal and its determinant is zero. Did you mean this one?

\[ \left [ \begin{matrix} -1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{matrix} \right ] \]

Corrected. Also did not know we could use latex in here.

One of the eigenvalues of the second matrix is -1, whereas all of the first matrix's eigenvalues are 1.

So I guess that orthogonal matrices have some sort of "sign" property to them, that gives the handedness of the basis?

Definitely no.

The math doesn't care where the thumbs are attached to your hands. "Handedness" comes into play when you visualize the results in real world, e.g. on the screen. The identity matrix always looks like your 1st matrix.

Your 2nd matrix performs a mirror operation. In general, if the determinant is negative, the transformation includes mirroring.

In this sense, there is a relationship between this "sign" and handedness: To project vectors from left- to a right handed coordinates, you can use a mirror operation.

These matrices ( A & B ) both qualify as being orthogonal.

[background=#f7f7f7]\[ \left [ \begin{matrix} 1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{matrix} \right ] \][/background]
[background=#f7f7f7]\[ \left [ \begin{matrix} -1 &0 &0 \\ 0 &1 &0 \\ 0 &0 &1 \end{matrix} \right ] \][/background]

So far, so good.

Furthermore, my code says their quaternion representation is actually the same ( both the identity quaternion).

I don't know what code you are referring to, but the second one cannot be represented by a quaternion, so things might not make a lot of sense.

glm gives me { 1/sqrt(2), 0, 0, 0 } for B curiously.

Perhaps the fact the quaternion you get doesn't have length 1 is an indication that something went wrong?

Also I know that det(A) = 1 and det(B) = -1

Correct.

So I guess that orthogonal matrices have some sort of "sign" property to them, that gives the handedness of the basis?

Yes. Orthogonal matrices are invertible, and the sign of the determinant of an invertible matrix is an important number. If you interpret the matrix as representing a change of basis, positive determinant means the bases have the same orientation.

And how this work with quaternions?

It doesn't. Unit-length quaternions can only represent rotations, whose determinant is 1.

I suspect it might have some relation to the famous 'double cover' property of quaternions that people always mention. Basically the fact that Q and -Q correspond to the same rotation.

I suspect you are wrong.

I also wonder if this works for complex numbers as well as they're the 2d equivalent of quaternions.

I am not sure what "this" is.

Are there any other properties involving this 'sign'? I have a vague feeling it had something to do with the diagonal as well.

Maybe someone here has some deeper insights or facts to share about this

No, sorry. I don't have anything deep to say about this.

EDIT: If you want to read something about it, this sign you are talking about is what separates the group O(3) into two halves, one of which is the subgroup of 3D rotations, SO(3).