I've finally figured out how to do portal occlusion culling with view frustum planes and portal AABB.

First the view frustum has planes in it that check for portal AABB.

The view frustum only checks for portals in the sector the player is in.

If a portal is in the frustum planes then render the next sector and stop.

Then the portal that is visible gets planes added to it, behind it and the planes follow the main camera.

The portal planes check for visible portals AABB in the sector it is connected to and stops after rendering the next sector.

This repeats until there are no more sectors with visible portals.

The view frustum portal checking stops at the sector the player is in and the portals do the check on the next sector and stop at the sector they are connected to.

This is way better than using raycasts for visibility.

It can be further simplified to a problem of 2D polygon clipping, so planes (or depth) are not even necessary.

The reason raytracing became mostly preferred is the unbounded complexity if the levels are quite open ended, opposed to levels form the quake days where a horizon was rarely visible. Developing a robust precalculation of PVS for such cases can become a nightmare pretty quickly. At this point it also seems easier to work with occluders instead portals, though occluders do not allow to use a low res rasterization, while portals do.