So, after about twenty years of hobby game development experience and fourteen years as a professional developer, I've finally decided it was time really understand the math. I don't consider myself good at math, and I haven't studied math in school since I graduated in 1999. So I'm rusty.

I've been working through the exercises at the end of the chapter of this book and I'm struggling with cross product. I understand how to calculate it, and I understand what it's used for, but not necessarily the math behind it. I understand it like a tool, but not in theory. I'm trying to use the cross product to determine the angle between two vectors.

The first part of the book's exercise:

Given the two vectors:

a = (0, 1, 1) 
b = (0, -1, 0)

Compute the angle between them using the scalar product. That's easy:

2.356

The second part is what I'm struggling with:

Using those same two vectors, calculate the angle between them using the cross product. The book seems to indicate that the following formula is used:

|| a x b || = ||a|| ||b|| sin(theta)

It goes on to explain that's the same as this:

|| a x b || = ||a|| ||b|| sqrt(1 - (a dot b)**2)

But this isn't what I'm coming up with. I'm calculating the magnitudes of the two vectors as:

||a|| = sqrt(0*0 + 1*1 + 1*1) = 1.414
||b|| = sqrt(0*0 + -1*-1 + 0*0) = 1

Cross product:

a x b = (a lot of typing) = (1, 0, 0)
|| a x b || = 1

Dot product:

a dot b = 0*0 + 1*-1 + 1*0 = 1

Then to plug the values into the formula:

1 = 1.414 * 1 * sqrt(1 - 1) = 0

Obviously this cannot be correct. I know there's some piece I'm missing, but I can't quite figure it out from the text. Could someone help me out here?

Wikipedia stated that the magnitude of the cross product of the unit vectors yields the sine, but even this isn't giving me the correct angle:

â = (0, 1, 1) / sqrt(0*0 + 1*1 + 1*1) = (0, 0.707, 0.707)
b? = (0, -1, 0) / sqrt(0*0 + -1*-1 + 0*0) = (0, -1, 0)
â x b? = (.707, 0, 0)
sin(theta) = ||â x b?|| = .707

So I thought inverse sine of .707 yield the actual angle, but it doesn't reproduce the same value I calculated with the scalar product (2.356):

arcsin(.707) = 0.785

But I have noticed that

arccos(-0.707) = 2.356

So what am I missing here? Please help!

Using those same two vectors, calculate the angle between them using the cross product. The book seems to indicate that the following formula is used:

|| a x b || = ||a|| ||b|| sin(theta)

It goes on to explain that's the same as this:

|| a x b || = ||a|| ||b|| sqrt(1 - (a dot b)**2)

This step assumes that cos(theta) = a dot b, but that is only true for unit vectors. For non-unit vectors, such as in you example, the correct equation is

|| a x b || = ||a|| ||b|| sqrt(1 - ((a dot b) / (||a|| * ||b||)**2))

Wikipedia stated that the magnitude of the cross product of the unit vectors yields the sine, but even this isn't giving me the correct angle:

â = (0, 1, 1) / sqrt(0*0 + 1*1 + 1*1) = (0, 0.707, 0.707)

b? = (0, -1, 0) / sqrt(0*0 + -1*-1 + 0*0) = (0, -1, 0)

â x b? = (.707, 0, 0)

sin(theta) = ||â x b?|| = .707

So I thought inverse sine of .707 yield the actual angle, but it doesn't reproduce the same value I calculated with the scalar product (2.356):

arcsin(.707) = 0.785

But I have noticed that

arccos(-0.707) = 2.356

So what am I missing here? Please help!

I believe this is a domain ambiguity. You are indeed getting the sine of the angle with the norm of the cross product, but the inverse sine is not necessarily the angle. You have two solutions to sin(x)=0.070, and you're getting one of them; the other one is 180-x. The inverse sine function is defined for -90 to 90 degrees, but your desired solution is at 135 degrees.

edit: The editor broke the code tags and tried to recreate them, in case some code boxes or their contents don't match the original post I quoted.

Only skimmed it, "a dot b = 0*0 + 1*-1 + 1*0 = 1"

Typo? = -1, not 1.