As far as I know, Metro Exodus has the most advanced working raytrace solution that can handle modern game polycounts. (Quake 2 RTX is “better” but the performance is extremely limited.)

I'm struggling to understand how raytracers can efficiently distribute samples, other than performing lighting for every single pixel on the screen. I have seen some info on “surfels” but I don't understand you would find the nearest surfels and interpolate between them, and the papers always list a lot of caveats to their usage.

To me, lightmapping seems like the only way to efficiently distribute lighting data without errors on geometry edges. You can easily control the distribution of pixels and because its all aligned to surface geometry there are never any problems with level-of-detail changes like voxels have, or object edges like downsampled screen-space lighting would have. It also would provide a very easy bridge between last-gen technology (baked lightmaps) and next-gen (continuously updating lightmaps using hardware raytracing).

What do you think?

Josh Klint said:
I have seen some info on “surfels” but I don't understand you would find the nearest surfels and interpolate between them

Ohhh, yeah… that's still my final open problem.

I wanted to do it like lightmaps. Each texel links to a surfel, and interpolation is easy too, at basically no cost.
A problem is texture seams. Because surfels (or any other approach of surface probes) are likely lower resolution than baked light maps, seams become very visible.
So i worked on tools to generate seamless UV maps, which is possible using quadrangulation, so texels always fit other texels across a seam. I found quadrangulation was the hardest problem i've ever worked on.
LOD is involved too. We need seamless, global parameterization on all LODs, and likely we even need a hierarchical mapping between the LODs.
Personally this brought me to the conclusion that lighting and LOD has to be solved at once, ideally using the same data structures for both.

I have this now, mostly, but it still feels not practical for general game models. Actually i require to resample the geometry, which can't preserve fine details of human made, technical objects.
Or, i would need to add extra edges to the original meshes, so the seamless UVs can be applied. But this amplifies geometry for no visual win and worse: LOD isn't solved then, again.

So that's why why >5 years after having the worlds fastest GI algorithm working, i still can't demonstrate or offer it for general application. It sucks.

Besides, even if we have such light mapping approach working, there is still another problem on top: With streaming open worlds and dynamic memory management for our surfels / probes, we need to update the indirection from probe index textures.
As this constantly changes, this update will have some cost, so maybe it's just better to have no direct mapping of surface to probe at all, but resolve it at runtime. Seems that's where i'll have to go.

I see those options (all are bad):

Make tiles from the GBuffer, for each tile traverse the surfel hierarchy and apply the found probes to the pixels of the tile.
It's not that bad, because we do not need a traversal for each single pixel.
But if we do ray tracing, then we need to do a full traversal per hitpoint to shade it (if offscreen).

Make a fast lookup acceleration structure for the surfels. E.g. a froxel grid. Then each pixel finds it's affecting surfels quickly.
We could extend this beyond the screen (to support RT) using a ‘cubemap’ of froxel grids, with an additional regular grid in the middle. I think EAs surfel approach did just that, proofing it's ‘fast enough’.

Splat the surfels to a screen sized buffer stochastically, so for each pixel, we iterate a small region of NxN texels to find all affecting surfels.
Maybe that's faster than building acceleration structures. But lacks support for RT again, ofc.

Edit: That's the ideas to map ‘screenspace to surfels’.
I forgot to mention alternatives, still aiming to prevent a need for a search:

High res indirection textures. Because of high resolution, seams would be no problem. But updating the texel pointers with streaming is an ugly brute force cost.

Surfel pointer per triangle. Tessellate geometry if needed so geometry resolution matches surfel resolution at least. Teh adjacent surfels can be found because i have adjacency pointers per surfel anyway, so no search.
This seems efficient if the surfel resolution is low enough (I was aiming for 10cm for PS4). But it requires to lock geometry resolution and so its LOD to the current surfel hierarchy LOD cut, or vice versa.

Not sure which evil to pick.
There are exceptions, where certain ideas become unpractical, like foliage.
So i could try to be efficient, probably requiring to have multiple techniques for different kinds of geometry,
or i could try to minimize complexity, using just one approach for all cases but requiring a search.

Josh Klint said:
What do you think?

It really, really sucks. <:/

But let me know if you have any other ideas…

btw, Godots SDF GI received some attention. I think it sucks, but well… currently anything else sucks as well: https://www.docdroid.net/YNntL0e/godot-sdfgi-pdf​

And here is paper about EAs surfels: http://advances.realtimerendering.com/s2021/index.html​

Edit: Basically the problem is the same as the many lights problem. Related works are ‘Clustered Shading’ or ‘Tiled Light Trees’. That's what i meant with froxel grid, but guess you know these works already.

The 2 published solutions here are both surface based. Hash grids are covered well by this AMD paper: https://gpuopen.com/download/publications/GPUOpen2022_GI1_0.pdf​

They're just gridlike approximations of surfaces, low res and inefficient from a bandwidth perspective, but dead simple and reasonably effective. Between this and surfels most programmers prefer this, mesh parameterization be damned (cause it is).

The other, far more complicated solution is Epic's surface cache, which builds something like a lightmap/material map in runtime: https://advances.realtimerendering.com/s2022/SIGGRAPH2022-Advances-Lumen-Wright%20et%20al.pdf​

It's got the advantage of being relatively high res and bandwidth efficient as you can compress the material map and radiance cache using DXT at runtime, realtime RT on consoles at good quality! The downside is it's massively more complicated than the above.

As for Metro, they've never talked about what they're doing afaik.

Frantic PonE said:
As for Metro, they've never talked about what they're doing afaik.

They have quite detailed explanation on their website or some A4 blog.
Iirc, they use volume grids of probes very similar to RTXGI. Probably very low spatial res.
But they only use it for the 2nd bounce. The rest is classical RT, so per pixel, and doing temporal / spatial denoising.
They did not mention any surface probes or caches beyond that.

Frantic PonE said:
The other, far more complicated solution is Epic's surface cache

Do you know their way to map those 'cards' to the actual geometry?
I guess it goes from their cards to the ‘screenspace probes’, and pixel sample this regular grid of SS probes.
But i still don't know how cards go to SS probes, or how they deal with areas of large depth complexity in screenspace, which then might map to only one SS probe, for example.
(I gave up on trying to understand their Lumen presentations :D )

Yeah the cards are kind of simple really, they're just kind of like an inverse cubemap. In the simplest case you take your six axis aligned cards from orthographic cameras pointing inwards to generate the cards, then on hit you project from hit to card to find your radiance.

The complications come in with stuff like concave objects and generating the cards quickly fundamentally relies on nanite. The hashgrid thing might be able to compress to BC6 with some cleverness though. Probably a good change to see in GI 1.0, among many others before it comes out.

Frantic PonE said:
Yeah the cards are kind of simple really, they're just kind of like an inverse cubemap. In the simplest case you take your six axis aligned cards from orthographic cameras pointing inwards to generate the cards, then on hit you project from hit to card to find your radiance.

Makes sense, thanks. I remember i've speculated this before. The limitation to convex objects is bad, but if you make your levels from modular models it should mostly work.
I wonder what they do for something like height map terrain. Maybe for such things they use volume grids of probes.

I'm not done yet with the AMD paper. Looking its images is not enough. Seems i'll have to actually read it… :D