I've gotten a portal thing to work ok by rendering the world from the other portal position to a texture, but I'm not sure which direction to actually render the view from. None of the solutions I can think of end up looking correct, like the portal is actually an extension of the room, and Google isn't very helpful.

I assume you are talking about holes in the wall, like in the Portal video game. Each portal has an associated reference frame. There is an affine transformation that maps one reference frame to the other one. You need to transform the camera through that transformation (or its inverse, depending on how you label things).

That's not very informative.

To render the scene from the other portals viewpoint, the camera position/rotation must be same relative to the other portal as it was to the first portal.

Of course the other portal will be facing out of the wall, while the first portal will be facing into the wall.

So if you camera is 5 meters in front of the first portal, it needs to be 5 meters inside the wall when rendering the scene from the other portals point of view.

This 'relative position/rotation' would normally be encoded in a 4x4 affine matrix and you would use matrix math to calculate the correct location for the camera.

The logic would be something akin to (portal1 and portal2 are matrices that hold the pos/rot of the portals):

Matrix relative = portal1:toObjectSpace(camera)
Matrix renderFrom = portal2:toWorldSpace(relative) //if portal2 faces out of the wall and portal1 faces into it, this works. If not, I think you need to transpose/inverse the relative matrix first

I dont know how to express that using 'proper' matrix notation. Probably something like

relative = camera * inverse(portal1)

renderFrom = portal2*relative

(renderFrom = portal2*inverse(relative) if portal2 is pointing into wall just like portal1)

But Im really not sure.

If you are not using matrices, you can use the same logic as in the first piece of code except instead of matrices you do the operations on position/rotation or whatever it is youre using.

Also since the camera will be inside the wall when rendering the scene from other portals PoV, you need to make sure nothing behind the portal gets rendered somehow (unless required for reflections or whatever).

If I have a floor and a wall next to it with the two portals on it, and I render the portal view from a distance behind the wall without the wall rendering, wouldn't I just see a floor far away, floating above the void?

I stitched together a quick graphic representation of the problem:

[attachment=28033:portal.jpg]

On the left is the room you're in, with the portal in orange and the player along with viewing direction in green.

On the right is the another room (I used two rooms for easier drawing)

Again, orange is the portal.

Purple is the camera for looking through into the room. it is the same distance and angle from the portal as the player, only that it is outside the room instead of inside it. The yellow lines represent the area of the second room you can see through the portal from your current position and rotation.

The further you're away from the first portal, the narrower the yellow area gets.

in the end, you'd need to render it in a way as if you were looking into the second room through a window the same size and shape as your portals.

I found this pdf about Narbacular Drop (Portal's predecessor):

https://www.digipen.edu/fileadmin/website_data/gallery/game_websites/NarbacularDrop/documents/narbacular_drop_technical_design_document.pdf

On page 19 they describe the steps on how to achieve this portal rendering

Right, I think what I was missing in my last reply was that only a subset of the image would be used, not the entire image. My question was originally what direction the portal cam would have to look if it's rendering at the portal pos rather than behind it (i.e. in the middle of the red horizontal line in the second room in your image), since that's the only part of it you'd be using anyway, but I guess I'll look into doing it this way if it's easier.