Quick question: does anyone know if directx RayTracing works with tesselation/displacement shaders? If yes, how does it interacts with the TLAS and BLAS? Just for curiosity sake ?

DXR doesn't "work with" those types of shaders, but what it does support is refitting and rebuilding of the BLAS that contains the geometry you want to ray trace. If you want to use tessellation to re-tessellate a mesh/model on a continual basis then you'll need to stream the geometry out to a buffer and issue a refit or rebuild of the BLAS before issuing the DispatchRays call later in the frame.

You can bind a UAV to any shader stage to output a post-tessellated index buffer and vertex buffer, insert a barrier and then issue the rebuild/refit. I haven't seen any sample code on how to do this, but I imagine you'll want to output from the Geometry Shader stage or use Stream Output (the former sounds preferable).

3 hours ago, Adam Miles said:

DXR doesn't "work with" those types of shaders, but what it does support is refitting and rebuilding of the BLAS that contains the geometry you want to ray trace. If you want to use tessellation to re-tessellate a mesh/model on a continual basis then you'll need to stream the geometry out to a buffer and issue a refit or rebuild of the BLAS before issuing the DispatchRays call later in the frame.

You can bind a UAV to any shader stage to output a post-tessellated index buffer and vertex buffer, insert a barrier and then issue the rebuild/refit. I haven't seen any sample code on how to do this, but I imagine you'll want to output from the Geometry Shader stage or use Stream Output (the former sounds preferable).

Thus I'm thinking a good solution for shadowing might be constraining tesselation amplitude and using screenspace shadowing/heightfield self shadowing. Precision issues can already be encountered with other raytracing effects,and be overcome with screenspace gap filling already, so this just seems like on more to add to the list.

Besides, tesselation already has texture stretching problems.