maybe the code itself will say more than 1000 words ill coment everything.

int index = model->VBO_BE[faceindex].INDEX_START; //first face vertex index in VBO
t3dpoint pop = model->VBO_V[index]; //point on checked plane

t3dpoint A = vectorAB(model->VBO_V[index],model->VBO_V[index+1]);  //vector from first vertex to second vertex of the plane (face)
t3dpoint B = vectorAB(model->VBO_V[index],model->VBO_V[index - model->VBO_BE[faceindex].length - 1]); //since its a quad this is a vector from first vertex to last vertex
t3dpoint n = Normalize(A * B); //calculate plane normal and normalize it




t3dpoint Ap = ProjectPointOntoPlane(n,pop, rA); //project ray start onto plane
t3dpoint Bp = ProjectPointOntoPlane(n,pop, rB);//ray end point on plane
t3dpoint AB = vectorAB(Ap,Bp); //make a vector on plane


float dstA = float(n3dPoint_plane_distance(rA, model->MATH_FACE_NORMALS[faceindex], model->MATH_FACE_DISTANCES[faceindex])); //oh yes since plane equations are Ax+By+Cz+D=0 i made helper arrays with these values (A,B,C,D)

float dstAB = n3ddistance(rA, rB);

float perDST = dstA / dstAB; //get perccentage of the distances of ray start from plane / distance between start and end of ray

t3dpoint PointOnPlane = Ap + AB*perDST; //since Ap should be on plane it should return actual intersection point
res = PointOnPlane;

helper functions

long double n3dPoint_plane_distance(t3dpoint p, t3dpoint vnormalnover, float D)
{
long double d = -1.0f;
long double num =  p.x * vnormalnover.x + p.y * vnormalnover.y+ p.z * vnormalnover.z +     D;
num = abs( num );
long double denom = vnormalnover.x*vnormalnover.x + vnormalnover.y*vnormalnover.y+vnormalnover.z*vnormalnover.z;
denom = sqrtl(denom);
if (denom == 0.0f) return -1;
return num / denom;
}


t3dpoint __fastcall ProjectPointOntoPlane(t3dpoint normal, t3dpoint pointonplane, t3dpoint projected_point)
{
t3dpoint v = vectorAB(pointonplane,projected_point);

float dist = Dot(v,normal);

return projected_point - (normal*dist);

}

What's your question?

its not returning porper point, well i would like to know if the math formula there is fine so i can check other things like (the ray itself, if is messing calculations)

since it has length of 7 000 000 (close to it),

i have another rayintersection function that returns good values however it returns intersection even when the ray shouldn't be doing it.

I must somehow gather all info so i can find the problem actually.

btw face itself is situated somewhere about 6 500 000 from 0,0,0 point, its an earth scale intersection i check for terrain collision i make a ray from 0,0,0 point to the object position and check if theres intersection

HISTORY:so heres a video of what i get (this red square is a face that spectator lies within) white (sometimes blue sphere) is face intersection point (its with the ol function i have)

i will spawn some zombies there and drop them from the sky so you will see that not all zombies catch the ground, so i decided to check that ray poly intersect function it says

theres a collision but with 4 different faces (instead of one) so i cant use it i decided to write my own (shown above but it doesnt even give me the proper point of ray plane intersection, so i ask if someone could check that code for errors :P)

Suggestion: Debugging a section of code is something you should learn to do in a methodical fashion. The comments you added to the code indicate you know what you want the code to do. The next step is for you to determine if the code does, in fact, do what you intended.

That approach is much faster and much more efficient than posting a couple dozen lines of code and saying "Here's some code. It doesn't work. What's wrong?" It's likely that you would've found the problem yourself in the last several hours, perhaps in a few minutes.

First, you have input data to your routine that only you can verify is correct. Your post asks other to assume your model data correct, the indexing is correct, you're accessing that data correctly, and your functions and overloaded operators do what you wrote them to do. As an example, it appears you've overloaded the multiplication operator vector*vector to calculate a cross product. There's no way for others to know if you wrote that correctly, or whether it was written to produce a right-hand or left-hand cross product.

So:

1. Set a breakpoint at the beginning of the section of code you think is not doing what you intend.

2. Run the program.

3. When the breakpoint is reached, examine values in each line of code in sequence, verifying that the input values are correct, and verifying that the resulting value or values are what you intend that line to produce.

4. Step to the next line of code and repeat the examination.

At some point, the input values will be incorrect or the output values will not be what you expect. Fix that line of code.

Start with a trivial quad as the input with a trivial raycast for which the results are easy to verify. E.g., a quad with corners at (1, 0, 1) and (-1, 0, -1), and a ray from ( 0, 1, 0 ) to (0, -1, 0). The plane normal should be (0, 1, 0) (assuming Y-up is what you expect from the winding order of the vertices), the distance should be 1 and the intersection (0,0,0). Those are all values easily verified by a quick examination.

If things work properly, move your ray away from the origin, and repeat. Do you get intersections within the quad limits at a Y value of 0?

Do you get the expected results if the ray is from (0, -1, 0) to (0, 1, 0)? For the ray (1,1,0) to (-1, 1, 0) [no intersection]?

Try quads at fixed positions parallel to various planes for which the normal is obvious and the intersection point is easily seen to lie within the quad.

pls buckeye dont eat me lol :P ok ill debug it

pls buckeye dont eat me lol ok ill debug it

I don't eat cats (very often).

well there was an error in the code i tried to read array[-3] index the function works fine for small numbers but it fails when i want to test a face that is sitouated 6 500 000 from 0,0,0 point

bool __fastcall RayIntersectsFaceFromModel(TachoGLModel * model, int faceindex, t3dpoint rA, t3dpoint rB, t3dpoint & res)
{
TPlane plane[4];

int index = model->VBO_BE[faceindex].INDEX_START;
t3dpoint pop = model->VBO_V[index];

t3dpoint A = vectorAB(model->VBO_V[index],model->VBO_V[index+1]);
t3dpoint B = vectorAB(model->VBO_V[index],model->VBO_V[index + model->VBO_BE[faceindex].length - 1]);
t3dpoint n = Normalize(A * B);




t3dpoint Ap = ProjectPointOntoPlane(n,pop, rA);
t3dpoint Bp = ProjectPointOntoPlane(n,pop, rB);
t3dpoint AB = vectorAB(Ap,Bp);

float dstA = float(n3dPoint_plane_distance(rA, model->MATH_FACE_NORMALS[faceindex], model->MATH_FACE_DISTANCES[faceindex]));

float dstAB = n3ddistance(rA, rB);

float perDST = dstA / dstAB;

t3dpoint PointOnPlane = Ap + AB*perDST;
res = PointOnPlane;
//	t3dpoint point,t3dpoint normal,float distance)


t3dpoint tA,tB,tC;


int i; 
for (i=0; i < model->VBO_BE[faceindex].length; i++)
{
tA = model->VBO_V[index+i];
if (i == model->VBO_BE[faceindex].length-1) tB = model->VBO_V[0];
else tB = model->VBO_V[index+i+1];

tC = tA + (n*1000.0);

plane[i].calc_plane(tA,tB,tC,false);
}

int side1 = plane[0].ClassifyPoint(PointOnPlane);
for (i=1; i < model->VBO_BE[faceindex].length; i++)

if (plane[i].ClassifyPoint(PointOnPlane) != side1) return false;

   return true;
}

struct TPlane {

double A;
double B;
double C;
double D;

double x;
double y;
double z;

t3dpoint n;
t3dpoint N;
t3dpoint tmp;

void calc_plane(t3dpoint p0, t3dpoint p1, t3dpoint p2, bool normalized)
{
						 n = Normal(p0,p1,p2, normalized);
if (normalized == false) N = Normalize(n);   else  N = n;
				//  n    = reverse_point(n);
x = (double)p0.x;
y = (double)p0.y;
z = (double)p0.z;

A = (double)n.x;
B = (double)n.y;
C = (double)n.z;

D = -(A*x + B*y + C*z);


}



int __fastcall ClassifyPoint(t3dpoint point)
{
float costam = dotproduct(point,n)+float(D);

if (costam > 0) return isFront;
if (costam < 0) return isBack;
if (betweenorequal(-SPECIAL_FEATURE,SPECIAL_FEATURE,costam) == true) return isOnPlane;

}


};

but it fails ...

At which line of code?

it doesnt throw any error, plane intersection point is situated outside of test face

t3dpoint Ap = ProjectPointOntoPlane(n,pop, rA);
t3dpoint Bp = ProjectPointOntoPlane(n,pop, rB);
t3dpoint AB = vectorAB(Ap,Bp);

float dstA = float(n3dPoint_plane_distance(rA, model->MATH_FACE_NORMALS[faceindex], model->MATH_FACE_DISTANCES[faceindex]));

float dstAB = n3ddistance(rA, rB);

float perDST = dstA / dstAB;

t3dpoint PointOnPlane = Ap + AB*perDST;


ProjectPointOntoPlane is shown in the first post

btw i made a mix of old function and new function (old calculates the ray plane intersection point) new one checks for collision and it works uuuuuuu :P

it doesnt throw any error, plane intersection point is situated outside of test face

Code needn't throw an error to be incorrect. Asking you which line of code causes the error was intended to urge you to find the problem, using the same technique outlined above.

If you have it working to your satisfaction, that 's okay. But, be aware: if you don't find the problem now, it will probably find you later.