I've been reading up Christer Ericson's Real-Time Collision Detection to implement GJK in my game (namely section 9.5.2), but I have some questions regarding how Ericson picks the correct face voronoi region that contains the origin.
In the book, he states that if the origin is not within any vertex or edge voronoi regions, then testing whether or not the origin is within any face voronoi regions simply becomes checking whether or not the origin and the last point from the tetrahedron are on the same side of the face plane. Let me elaborate:
Let's define the tetrahedron Q as a set of four points, Q = {Q1, Q2, Q3, Q4}.
Say we are currently testing face Q1Q2Q3, and we define origin as point P. We can obtain the normal of face Q1Q2Q3 by cross producting two edges, I will denote the normal as N now.
The plane that contains Q1Q2Q3 divides the 3D space into two halfspaces. Ericson's face region test is to check if Q4, the only other point from the tetrahedron, and P, the point we are checking against, reside in two different halfspaces.
Since the plane is defined by its normal N, we can do dot(P - Q1, N) and dot(Q4 - Q1, N) and check if they have different signs (that means they are in two different halfspaces). Here's the face region check written out in code:
if (dot(P - Q1, N) * dot(Q4 - Q1, N) < 0) PIsInFaceRegionQ1Q2Q3();
However, I have an issue with this approach. Consider this tetrahedron:
It is obvious that the origin is contained by the face voronoi region made up by red vertex, yellow vertex, and green vertex. However, if I take a picture of it from the side, we can see that this tetrahedron is nearly coplanar:
'
If we do Ericson's test for face made up by red, yellow, and green vertex, then the test tells us that, yes, origin is inside RYG's voronoi region. However, if we do the same test on the face made up by green, yellow, and blue vertex, the test tells us that the origin is ALSO in GYB's region. The test result becomes ambiguous. The origin can be simultaneously in different halfspaces for two different triangle faces of the same tetrahedron while the origin is not in any vertex region or edge region.
Therefore, I think either this technique is wrong or I am not understanding it correctly. Am I correct on saying that this way of testing face voronoi regions is wrong? If so, what is the best way to do it then? Thanks a lot!