I have four points in 2D space that make up a rectangle, top left, top right, bottom left and bottom right. Given the x and y of an arbitrary point I want to know where that lies in the rectangle. eg, if the point happens to lie on the top left corner it'd be (0,0) and if it happens to lie an the bottom right corner it'd be (width,height). I don't have a rotation matrix for said rectangle but I could make one from the points. I just thought there should be a much simpler way but my mind is having a blank.

Hmm. Actually, it could be a trapezoid based on the position of the camera.

I have four points in 2D space that make up a rectangle, top left, top right, bottom left and bottom right. Given the x and y of an arbitrary point I want to know where that lies in the rectangle. eg, if the point happens to lie on the top left corner it'd be (0,0) and if it happens to lie an the bottom right corner it'd be (width,height). I don't have a rotation matrix for said rectangle but I could make one from the points. I just thought there should be a much simpler way but my mind is having a blank.

I guess it'll be better if you just make the translation and rotation of the rectangle and the point, and then a simple substraction will give you the result.

If you really want to avoid that you can define two lines in space, one from top-left to top-right and the other from top-left to bottom left. Then, compute the distance of the arbitrary point to each of those lines. Those distances will be the result you want: http://en.wikipedia.org/wiki/Distance_from_a_point_to_a_line

Anyway, the second method looks more complex than the first one with those squared roots and power of 2, and the need of compute the lines of the rectangle.

Hmm. Actually, it could be a trapezoid based on the position of the camera.

What are you trying to do? Is it a 2D or 3D game? If the camera position makes a rectangle look like a trapezoid then I guess it's 3D, in which case translation and rotation matrices will make everything easier for future stuffs.

It's a 3D game but the problem is 2D. To cut a long story short, I'm having to pass four uniforms into a pixel shader for the 2D screen space positions of the four corners of a shape. This will end up being a rectangle if the camera is head on, otherwise it'll be a trapezoid. For each pixel in the shader I have the screen space position and I want to know 1) if the pixel is within those four coordinates and if so 2) calculate the amount across and up the shape for texture lookup.

Actually, I might be in luck. It looks like the camera will always be head on, which will make the shape always a rectangle. I imagine that makes things easier?

It's a 3D game but the problem is 2D. To cut a long story short, I'm having to pass four uniforms into a pixel shader for the 2D screen space positions of the four corners of a shape. This will end up being a rectangle if the camera is head on, otherwise it'll be a trapezoid. For each pixel in the shader I have the screen space position and I want to know 1) if the pixel is within those four coordinates and if so 2) calculate the amount across and up the shape for texture lookup.

It sounds a lot like you're just trying to draw a quad? How about instead of sending 4 points as uniforms, send 4 points as vertices with texture coordinates, which will get transformed in the vertex shader and rasterized in the pixel shader. The texture coordinates will be interpolated by the hardware so you won't have to do any math to determine them. And all the pixels that get rasterized will, by definition, be within the rectangle, so you won't have to perform any tests for that, either.

Yeah, that was part of the long story short. In this particular example, because of the way the pipeline works doing that would require some refactoring that I didn't want to do. You're right, though. That is the proper way to do it. I should just bite the bullet and do the refactoring instead of being lazy :)

Yeah, that was part of the long story short. In this particular example, because of the way the pipeline works doing that would require some refactoring that I didn't want to do. You're right, though. That is the proper way to do it. I should just bite the bullet and do the refactoring instead of being lazy

Haaa -- I know that one! It is so easy to be lazy in that way that you end up spending so much more time to get things done than if you had just done the right thing right the first time!

I know, because I have been there soooo many times. ;)

Edit: And it feels so much better when you do end up doing things the way you really want to - the proper way - and you know that your code is another small notch better than it was earlier.