Hi,

I've implemented Occlusion culling by selecting a number of big occluders and extracting planes from their AABB, forming an occlusion volume. In principe this works, but the thing is that the near plane(s) I'm using have a ‘d’ based on the front of the occluder instead of the back, meaning that the intended culling only happens when the camera is super close to the near plane(s). I've added some screenshots to illustrate:

1. view before constructing the debug drawn edges of the planes
2. hiding the occluder’s meshrenderer and debug draw the 5 planes (in this specific case)
3. moving closer to the near plane, stuff gets occluded
4. moving to almost the exact edge of the near plane
5. occluder planes visualized (the edges), pDebugPoints in the code

This is the code I'm using: https://pastebin.com/DkdaGAx2.

I'm basically trying to find a way to let the near plane(s) distance (plane.d) be based on the rear of the occluder instead of the front. Any ideas on how I could achieve that?

cozzie said:
I'm basically trying to find a way to let the near plane(s) distance (plane.d) be based on the rear of the occluder instead of the front. Any ideas on how I could achieve that?

Just some ideas:

You could use a special pixel shader for back facing triangles and modify the depth value to be the near clipping plane. (But changing depth disables HW optimizations and has a cost.)
Maybe there is a better trick for that, but idk.

You could do polygon clipping on the box to find the polygon which cuts it at the near plane, then render this polygon together with the box. (Similar to the general frustum clipping done in software rendering.)
Could run entirely on GPU with compute shaders generating indirect draws.

cozzie said:
but the thing is that the near plane(s) I'm using have a ‘d’ based on the front of the occluder instead of the back, meaning that the intended culling only happens when the camera is super close to the near plane(s).

I'm confused. I assume if you render both front and back faces of the occluder box, culling should work no matter if the near plane cuts a hole into the front faces or not. The back faces should still cull the buildings behind them, and failure cases should only affect the geometry inside the box, e.g. the rooms inside the building which gives the box.

Thanks for the reply @joej . The thing is that this is all happening on CPU side, not perse about rendering and depth read/writes.

I just take the worldspace AABB and the camera position, and using those, I construct N occlusion planes for the AABB, defining the invisible volume. So I can do plane/AABB (or sphere) intersects with groups of objects against that volume (and discard them from drawing).

The occluder box is normally rendered ‘as usual’, as there's no depth buffer involved, I just take the 8 corners of the AABB in worldspace, to construct the planes.
I'm basically looking for a way to ‘push’ the volume further so the occluder's volume is not part of what's rejected, which I believe is now the case, because there's volume of the occluder itself, included (if that makes sense 🙂)

Now i'm even more confused : )
It's it all on CPU, but then you also say occluder boxes are rendered.

Do you try to cull what's behind the box, e.g. other buildings?
Or do you try to cull what's inside the box, e.g. interiors of the building?

I ask becasue assuming it's the former, plane tests from a single box would not help in this case (top down view):

Neither the blue nor the orange occluder boxes can cull the red building behind them, because they occlude only parts of it.
Only if we can merge the occlusion of both boxes the red box can be culled.
The usual way to do this is rasterizing the occluder boxes, either on GPU and then using occlusion queries, or on CPU with z spans for example.

You would need to explain better, as ther are so many forms of ‘occlusion culling’ with varying culling efficiency vs. implementation complexity.

cozzie said:
I'm basically looking for a way to ‘push’ the volume further so the occluder's volume is not part of what's rejected, which I believe is now the case, because there's volume of the occluder itself, included (if that makes sense 🙂)

I guess you mean this:

The red bounding box of the house can't be used as an occluder, we actually want the blue box?

That's a hard problem ofc., often leading to making occluder geometry manually.

As mentioned when you were implementing this, you can't do anything to optimize the near plane as things stand. You first need the interior volume and not the exterior AABB. If you try to optimize using the exterior AABB it will simply introduce false positives, near plane must remain at the furthest point of the AABB until you get the correct data (and you are still going to have false positives). Second, once you have the interior volume via hand crafted AABB/OBB or something that generates it, you can only pull the near plane towards the camera based on the ‘furthest’ point of the visible silhouette edges. Anything which pulls the near plane closer will again introduce false possitives.

Basically, at this time, unless you've fixed it, you are working with bad data and the old rule: “bad data in == bad data out” is in effect. So, trying to optimize this further is just likely to exaccerbate the current errors. An no, having just reduced the size of the AABB isn't gonna work, as mentioned it was just a stop gap to make it kinda look more correct. :P So, once again, stop optimizing it till you get the data fixed, then you might not care enough to bother because it ends up being a fairly trivial gain.

Thanks

@all8up what's the definition of the interior AABB? is that basically the volume that FULLY inside the building? If so, I think the result with the ‘hacky decreased exterior AABB’ would give the same result, wouldn't it? Not saying I don't need a separate interior AABB btw (I do need it), but I don't understand how that would change anything on culling to late because of the near plane(s).

@cozzie Yes, the interior AABB would be an AABB which can not be seen from any externally visible view of the object. Take a circle as an example, if you use the AABB of that as an occluder and view it from a 45 degree angle, you are going to be occluding a huge amount extra area that should be visible (i.e. one of the AABB corners will stick way out). The proper solution for the circle is to build the ‘inscribed’ AABB (related to: https://mathcentral.uregina.ca/QQ/database/QQ.09.06/s/benneth1.html ). It's basically the same problem. Most folks do this by hand because it is a difficult problem when it comes to arbitrary geometry. It “can” be done, usually via voxelation, but generally is a lot of work for something an artist can do in less than a minute.

Thanks @all8up . The interior AABB sounds like an AABB that would completely fit/ be encapsulated inside the exterior AABB. Which is what I did manually now with the ‘primary AABB’, which obviously causes other issues with culling, collision etc. All clear that there needs to be a separate (interior) AABB.

What's unclear for me still though, is how this would solve the ‘late culling’ only when I'm at the edge of the AABB. I tried decreasing the size of the AABB, but then the effect is as ‘inverted’, as is it gets worse 🙂 Which could imply that I need a bigger AABB, but more likely a different way to define the near plane(s).

I also recorded a video to show the current output. Basically what gets culled when standing at the edge of the greenAABB, should be culled when you're at the camera position at the end of the video.

So, it looks like you have a bad facing with some plane somewhere. Consider that the case in the video should have only 4 bounding planes (aka, only one face visible so 4 silhoutte edges) and a capping plane which in this case should be the visible plane you are looking at. Remember that we took some very liberal short cuts (aka: hackery) to deal with the fact that you had bad data at the time. Given that, we also punted on the correct way to get what you keep calling the ‘near plane’ which is incorrect, it is just a cap which is not neccessarilly perpindicular to anything in the occlusion volume. We only did the single plane as an optimization and because it was a non-issue until you had the other bits functional.

To revisit with an ugly and noisy image:

Your current issue is pretty likely related to that ‘near’ plane, but lets clarify things a bit. The ‘near’ plane we were supposed to be computing was just a plane perpindicular to a ray formed from the eye point to the approximate center of the silhoutte n-gon and it would be placed at the furthest silhoutte point distance from the eye point. I.e. the orange line. But, in reality, once the silhoutte extraction was correct, you should use the green plane as the volume cap by using the 3 points (3D version of the above image) on the silhoutte points which are furthest away from the eye point.

Now, having said all that, stepping “into” the occlusion volume and having things change the way they are is a problem in itself. Those planes are always supposed to face into or out of the volume, either one as long as it is consistent and you change the dot comparison. In fact, as soon as you enter the AABB, occlusion should turn off as it becomes an invalid occluder. I.e. you can't “occlude” with something you are not viewing from the outside.

Long story but, check how you are finding the normals for the planes, it seems like it is wrong. Also, fix that volume cap to be correct now that “most” things are technically working.

Thanks @all8up

• in the case of the video, I end up with the following planes
  • find quads facing camera, in this case 1, resulting in 1 plane
    • normal facing the center of occluder
  • from each quad facing the camera, store the edges
    • and remove any duplicate edge (if duplicate, remove both)
  • resulting in 4 planes
    • one for each 2 verts making up the edge + the camerapas
    • the normals of the planes are all facing towards the center of the AABB
  • so total 5 planes
• this: “you should use the green plane as the volume cap by using the 3 points (3D version of the above image) on the silhoutte points which are furthest away from the eye point.”, I'm currently not doing. Especially when >1 quads face the camera, this is extra relevant
  • Q; how would I do this?

Here's the code I use to calculate the planes. Tbh I think the ‘side’ planes are actually fine, the issue seems to be just with what you called the ‘cap’ / aka near plane.
https://pastebin.com/q9Un3sp2

Curious if you spot anything weird in calculating the planes (pastebin) + perhaps have some pointers on the question above (using 3 furthest away silhouette points).