Okay so I know this has been asked before and it seems to be answered with DirectX or OpenGL terminology, which frankly makes me confused. I'm looking to solve this problem with python.

Anyway, I have a depth map that I created in Blender. A simple black and white gradient denoting distance from the camera per pixel.

I also have the intrinsic matrix for the camera.

From what I understand, the intrinsic matrix translates world coordinates to a 2D projection matrix (the screen). I think this is correct. Please correct me if I'm wrong. With this in mind, can you can take the (u, v) depth value from a pixel in the depth map and use values from the intrinsic matrix to find that pixels position in world space?

If so, what values in the matrix would they be and what operation (s) would you use?

I guess if you look at it from a simplistic “projective geometry” point of view, the depth map is depth value resulting from a projection of a world space position onto the image plane. If the intrinsic matrix represents this transform from world space to image space, then applying the inverse of the intrinsic matrix to an image space position( along with the associated depth) would get you back to world space.