Here is the source code I had written a while back and just gave up on because it wouldn't work. I don't know how to make the projection matrix have the reverse effect I mention above in my other post.
/* This function should create a point in world space that will render right on top of your mouse cursor This function, when it's working, is good for selecting objects in world space in perspective mode.*/S3Dvoid s3d_camera::ScreenToWorld(S3Dfloat xin, S3Dfloat yin, S3Dfloat zin, S3Dfloat* xout, S3Dfloat* yout, S3Dfloat* zout){ S3Dmat16f CameraMat; S3Dmat16f CameraInverseMat; S3Dmat16f ProjectionInversMat; S3Dvec3f VecUp; S3Dvec3f VecForward; S3Dvec3f VecRight; S3Dvec4f Pos; S3Dint ViewPort[4]; glGetIntegerv(GL_VIEWPORT, ViewPort); Pos[3]=1.0f; //flip y yin=ViewPort[3]-yin; //orientate point relative to the center of the viewport xin=xin-(ViewPort[2]*0.5f); yin=yin-(ViewPort[3]*0.5f); //retrieve camera information GetPosVector3f(Pos); GetRightVector3f(VecRight); GetUpVector3f(VecUp); GetForwardVector3f(VecForward); //set up vectors that will eventually be added to Pos s3dVecMultiplyScalar3f(VecRight, xin); s3dVecMultiplyScalar3f(VecUp, yin); s3dVecMultiplyScalar3f(VecForward, zin); //orientate Pos s3dVecAdd3f(Pos, VecRight); s3dVecAdd3f(Pos, VecUp); s3dVecAdd3f(Pos, VecForward); /* I probably need to add the near plane distance to Pos too...I dunno yet */ glGetFloatv(GL_PROJECTION_MATRIX, ProjectionInversMat); /* I NEED TO INVERT THE PROJECTION MATRIX HERE. */ //invert the camera matrix s3dMatCopy16f(CameraInverseMat, CameraMatrix); s3dMatInvert16f(CameraInverseMat); //move the Pos into camera space s3dMatMultiply4x16f(Pos, CameraInverseMat); //do a reverse projection s3dMatMultiply4x16f(Pos, ProjectionInversMat); //move the Pos back into world space s3dMatMultiply4x16f(Pos, CameraMatrix); *xout=Pos[0]; *yout=Pos[1]; *zout=Pos[2];}
Entering 0.0f in zin should result in a point lying right on the near plane. Entering the far plane distance into zin should result in a point perpendicular to the view while also lying directly on the far plane. xin and yin are the x and y position of the mouse cursor in window coordinates.
[edited by - WhatEver on January 19, 2003 9:29:17 PM]