I added doom 3 style touchscreens to my game engine; looking at the screen from a particular angle/distance makes a cursor appear on screen. clicking on the screen (or smacking it with a baseball bat!) interacts with it. Interactions are simple here, but you can write each screen to handle interactions in 2D coordinates, so you can do anything you want with it.

Here's how I calculated the position of the mouse on the screen

1. Get the rotated normal vector of the screen (notice in my gif that the screen is rotated upward slightly)
2. Get the cross product of the rotated normal and a constant “up” vector ((0, 1, 0) in DirectX's lefthand coordinates). Call this vector xAxis
3. Get the cross product of xAxis and the rotated normal. call this yAxis
4. Use regular raycasting to get the intersection of the camera's forward vector and the quad representing the screen. This point is in world space. Call this point hit.
5. Subtract hit from the origin of the screen (which is at the top left corner of the quad, for me) to get the offset of the intersection in model space. Call this point pos.
6. Project pos onto yAxis, then get the length of this vector. This will give you the length of pos along yAxis. Call this value yModel.
7. Project pos onto xAxis, then get the length of this vector. This will give you the length of pos along xAxis. Call this value xModel

xModel and yModel are effectively the “coordinates” of the cursor on the screen in model space. Now you need to convert these lengths into pixels that match the resolution of your target texture. For me, a 400x400 pixel screen texture results in a 1x1 unit quad in world space. The screen you see in the gif above is 128x128. So, to convert from model space to target texture resolution, I simply need to multiply x and y by 400 (i.e: a vector of length 0.5 will be 200px, and a vector of length 0.12 will be 48px, etc).

Here's my C++ code if this wasn't very clear to you:

        /* buffer is an array of all objects returned by the raycast query, in PhysX */
        if (GAME->tryRayCast(cameraPos, XMVector3Normalize(cameraLook), &buffer, 1000.0f, PLAYER->getFilter()))
        {
            for (i32 i = 0; i < buffer.getNbAnyHits(); i++)
            {
                auto _hit = buffer.getAnyHit(i); /* hit is the position of the intersection in world space. It's called _hit here because it's still a PhysX vector, not a directX vector */
                auto actor = _hit.actor; /* actor is the physics container being collided with. In this case "this" is the "menu" game object. */
                if (actor && actor->userData == this) /* userdata is the game object associated with the phyics container. This check means the raycast intersected the menu */
                {
                    auto xAxis = XMVector3Cross(XMVectorSet(0.0f, 1.0f, 0.0f, 0.0f), rotatedNormal); /* vector pointing in the direction of the xAxis of the screen */
                    auto yAxis = XMVector3Cross(rotatedNormal, xAxis); /* vector pointing in the direction of the yAxis of the screen */
                    auto hit = PxToXM(_hit.position); /* convert from PhysX to DirectX */
                    auto pos = hit - m_pos; /* convert from worldspace to model space. m_pos is the origin of the screen */

                    auto y = XMVectorGetX(XMVector3Length(XMProject(yAxis, pos))) * 400.0f; /* project pos onto yAxis, get the length, then multiply by 400 */
                    auto x = XMVectorGetX(XMVector3Length(XMProject(xAxis, pos))) * 400.0f; /* project pos onto xAxis, get the length, then multiply by 400 */
                    
                    y = m_height - y; /* convert y coordinate so the origin is at the top left of the screen */

                    at = make_pair(x, y);  /* at is the final coordinates. it is an std::pair<f32, f32> */
                }
            }
        }

Also sorry for this gif but this forum doesn't seem to allow me to embed a .mp4 file. You may notice the cursor appears to “lag” in the recording, but for some reason I only get this when I'm recording my screen, it looks perfectly fine when it's not being recorded (o゜▽゜)o☆