Following scenario:

I have a moving AABB and I want to find the collision point with a convex polygon in 3D space.

Finding the collision point with the plane of the polygon was fairly easy, however I'm unsure how I would go about checking whether the AABB is actually within the bounds of the poly.

If it were a point instead of the AABB I could simply check the dot product towards each edge, but I obviously can't do that in this case.

Any suggestions?

Are you after speed or precision?

- precision: convex-convex polytope intersection can be tested by testing if a vertex of object A is behind all faces of object B. The test needs to be done both ways. If the objects' bounding volumes intersect, but no vertices are contained, a more elaborate (and costly) edge-polygon intersection pass must be made (again both ways). You can exit as soon as you encounter the first positive result. If all vertices of an object are behind the other object's faces, then the first object is wholly contained; note that the opposite is not, true, though and the test needs to be run with the objects swapped to get correct results.

- if you want optimal speed, but are willing to give up some precision, you can run a kDOP-kDOP intersection pass on objects' bounding planes. It's much cheaper, but depending on complexity, a kDOP may not necessarily wrap your concave mesh perfectly. Note that your cube is a 6-DOP and you can use an n-DOP for your convex mesh. Here's the code for a k-DOP/k-DOP intersection test (from Realtime Collision Detection):

int TestKDOPKDOP(KDOP&a,KDOP&b, int k){
//Check if any intervals are non-overlapping, return if so
for(int i =0; i<k/2; i++)
if(a.min>b.max ||a.max<b.min)
return0;
//All intervals are overlapping, so k-DOPs must intersect
return 1;
}