Hi

I'm (still) strugling with adding DXR into my engine
Now when all the infrastructure is in place I'm trying to launch it (NVidia 2080, Latest Windows 10, latest nvidia drivers, all DXR examples and DXR implementation in UE4 works just fine)

I cannot create ID3D12StateObject

Here is the dump of object in question with comments (marked --->) what is what

--------------------------------------------------------------------
---> [This is my library, and mapping of library into unique names, since in future I can have multiple libraries with that same names]
| D3D12 State Object 0x0000004AB897D238: Raytracing Pipeline
| [0]: DXIL Library 0x000001B1CA8A0100, 18500 bytes
|  [0]: ?RayGeneration@@YAXXZ --> RG_010C79FD_D1D898AD_?RayGeneration@@YAXXZ
|  [1]: ?RayHit@@YAXURayPayload@@UBuiltInTriangleIntersectionAttributes@@@Z --> RH_010C79FD_D1D898AD_?RayHit@@YAXURayPayload@@UBuiltInTriangleIntersectionAttributes@@@Z
|  [2]: ?RayMiss@@YAXURayPayload@@@Z --> RM_010C79FD_D1D898AD_?RayMiss@@YAXURayPayload@@@Z
|--------------------------------------------------------------------
---> [This is my hit group]
| [1]: Hit Group (Hit_15D00DA7_C3757D0A)
|  [0]: Any Hit Import: [none]
|  [1]: Closest Hit Import: RH_010C79FD_D1D898AD_?RayHit@@YAXURayPayload@@UBuiltInTriangleIntersectionAttributes@@@Z
|  [2]: Intersection Import: [none]
|--------------------------------------------------------------------
---> [This is my local RootSignature]
| [2]: Local Root Signature 0x000001B1CABCB5F0
|--------------------------------------------------------------------
---> [That is assigned to my HitGroup]
| [3]: Subobject to Exports Association (Subobject [2])
|  [0]: Hit_15D00DA7_C3757D0A
|--------------------------------------------------------------------
---> [This is my local RootSignature]
| [4]: Local Root Signature 0x000001B1CABCB5F8
|--------------------------------------------------------------------
---> [That is assigned to my RayGen shader]
| [5]: Subobject to Exports Association (Subobject [4])
|  [0]: RG_010C79FD_D1D898AD_?RayGeneration@@YAXXZ
|--------------------------------------------------------------------
---> [This is my local RootSignature]
| [6]: Local Root Signature 0x000001B1CABCB600
|--------------------------------------------------------------------
---> [That is assigned to my RayMiss shader]
| [7]: Subobject to Exports Association (Subobject [6])
|  [0]: RM_010C79FD_D1D898AD_?RayMiss@@YAXURayPayload@@@Z
|--------------------------------------------------------------------
---> [This is Raytracing config, sizeof(RayPayload) == 64, attributes are 2 floats (barycentrics) so I pass 8]
| [8]: Raytracing Shader Config
|  [0]: Max Payload Size: 64 bytes
|  [1]: Max Attribute Size: 8 bytes
|--------------------------------------------------------------------
---> [This associates HitGroup, RayGen and RayMiss shader to Raytracing config]
| [9]: Subobject to Exports Association (Subobject [8])
|  [0]: Hit_15D00DA7_C3757D0A
|  [1]: RG_010C79FD_D1D898AD_?RayGeneration@@YAXXZ
|  [2]: RM_010C79FD_D1D898AD_?RayMiss@@YAXURayPayload@@@Z
|--------------------------------------------------------------------
---> [This is Pipeline config with recursion set to 2]
| [10]: Raytracing Pipeline Config
|  [0]: Max Recursion Depth: 2
|--------------------------------------------------------------------
---> [This is global root signature for all shaders]
| [11]: Global Root Signature 0x0000004AB897CA28
|--------------------------------------------------------------------
---> [This is empty local signature *(1)]
| [12]: Local Root Signature 0x0000004AB897D130
|--------------------------------------------------------------------

*(1) According to NVIDIA the current implementation of DXR requires pipelines to contain at least one global and one local empty root signatures, which do not have to be associated with any shader.

D3D12_STATE_OBJECT_DESC StateDesc;
StateDesc.NumSubobjects = UINT(StateSubObjects.size());
StateDesc.pSubobjects   = &StateSubObjects[0];
StateDesc.Type            = D3D12_STATE_OBJECT_TYPE_RAYTRACING_PIPELINE;

GetDevice()->CreateStateObject(&StateDesc, __uuidof(ID3D12StateObject), (void **)&StateObject)

And here is error message - witch is not very helpfull

D3D12: Removing Device.
D3D12 WARNING: ID3D12Device::RemoveDevice: Device removal has been triggered for the following reason (DXGI_ERROR_DRIVER_INTERNAL_ERROR: There is strong evidence that the driver has performed an undefined operation; but it may be because the application performed an illegal or undefined operation to begin with.). [ EXECUTION WARNING #233: DEVICE_REMOVAL_PROCESS_POSSIBLY_AT_FAULT]

Ofcourse all Root signatures are legit, the library is legit, etc.
Is there something aparrently wrong with my D3D12_STATE_OBJECT_DESC ?
How to debug this kind of problem, since the D3D12_STATE_OBJECT_DESC is realy simple, and 'should just work or spill anything usefull out of log'
(I'v tried with both state #12 added, and without - no difference at all)

I'v also exported working ID3D12StateObject from NVidia example
and now I have doubts about how it works

--------------------------------------------------------------------
| D3D12 State Object 0x0000006DDABBE928: Raytracing Pipeline
| [0]: DXIL Library 0x0000022A5A7DCF00, 6356 bytes
|  [0]: rayGen
|  [1]: miss
|  [2]: planeChs
|  [3]: triangleChs
|  [4]: shadowMiss
|  [5]: shadowChs
|--------------------------------------------------------------------
| [1]: Hit Group (TriHitGroup)
|  [0]: Any Hit Import: [none]
|  [1]: Closest Hit Import: triangleChs
|  [2]: Intersection Import: [none]
|--------------------------------------------------------------------
| [2]: Hit Group (PlaneHitGroup)
|  [0]: Any Hit Import: [none]
|  [1]: Closest Hit Import: planeChs
|  [2]: Intersection Import: [none]
|--------------------------------------------------------------------
| [3]: Hit Group (ShadowHitGroup)
|  [0]: Any Hit Import: [none]
|  [1]: Closest Hit Import: shadowChs
|  [2]: Intersection Import: [none]
|--------------------------------------------------------------------
| [4]: Local Root Signature 0x0000022A5ADCB2E0
|--------------------------------------------------------------------
| [5]: Subobject to Exports Association (Subobject [4])
|  [0]: rayGen
|--------------------------------------------------------------------
| [6]: Local Root Signature 0x0000022A5ADCBCA0
|--------------------------------------------------------------------
| [7]: Subobject to Exports Association (Subobject [6])
|  [0]: triangleChs -------------------------------------------------> ?? Why the local root signature is bound to 'Closest Hit Import' not to the 'TriHitGroup' hitgroup ? this is even legit ?
|--------------------------------------------------------------------
| [8]: Local Root Signature 0x0000022A5ADCAB90
|--------------------------------------------------------------------
| [9]: Subobject to Exports Association (Subobject [8])
|  [0]: PlaneHitGroup -----------------------------------------------> The another local root signature is bound to the hitgroup
|--------------------------------------------------------------------
| [10]: Local Root Signature 0x0000022A5ADCBF10
|--------------------------------------------------------------------
| [11]: Subobject to Exports Association (Subobject [10])
|  [0]: miss
|  [1]: shadowChs ---------------------------------------------------> ?? Yet another one is again bound just to 'Closest Hit Import'
|  [2]: shadowMiss
|--------------------------------------------------------------------
| [12]: Raytracing Shader Config
|  [0]: Max Payload Size: 12 bytes
|  [1]: Max Attribute Size: 8 bytes
|--------------------------------------------------------------------
| [13]: Subobject to Exports Association (Subobject [12])
|  [0]: rayGen
|  [1]: miss
|  [2]: triangleChs -------------------------------------------------> ??? Documentation states that 'Raytracing Shader Config' should be bound to RayGen, RayMiss and HitGroups ... but the example binds it to 'Closest Hit Import' of HitGroup
|  [3]: planeChs
|  [4]: shadowMiss
|  [5]: shadowChs
|--------------------------------------------------------------------
| [14]: Raytracing Pipeline Config
|  [0]: Max Recursion Depth: 2
|--------------------------------------------------------------------
| [15]: Global Root Signature 0x0000022A5ADCBA30
|--------------------------------------------------------------------

why in some cases the example binds LocalRootSignature to MEMBER of hitgroup, yet another time it binds it to HitGroup ?

I still see no reason why my structure does not produce valid object, and the example produces one

I have no idea if this is related to your issue, but you shouldn't need to explicitly associate your raytracing config and pipeline config sub-objects to your shader exports. After browsing through the spec I think it's legal to explicitly associate them, but there's not really a point in doing so since your configs need to be the same across the whole state object anyway. So you should be able to take out the sub-object association if you want, I've certainly never used one for associating things to the configs. Perhaps there's a driver bug with having that association in the there, so it's probably worth a try to take it out.

As for the Nvidia example where it's associating things to individual shaders within a hit group (instead of the hit group itself), that's legal according to the spec.

?RayGeneration@@YAXXZ

OMG!!! after 3 hours of stripping down the bits of code and trying various thigns I'v finally managed to figure out what was wrong!!!

Behold - there is a solution

In my Library definition I'v filled D3D12_EXPORT_DESC::ExportToRename with _MANGLED_ name (as it came from reflection D3D12_FUNCTION_DESC::Name) and D3D12_EXPORT_DESC::Name with the name + library name.

Filling D3D12_EXPORT_DESC::ExportToRename with mangled name (?RayGeneration@@YAXXZ) was the root of the driver crash (WTF - seriously - WTF!)

If the D3D12_EXPORT_DESC::ExportToRename is set to non-existing export, then the debug layer reports error - OK
if the D3D12_EXPORT_DESC::ExportToRename is set to non-mangled name (for ?RayGeneration@@YAXXZ it would be RayGeneration) then the ID3D12StateObject will be created
if the D3D12_EXPORT_DESC::ExportToRename begins with '?' character (?RayGeneration@@YAXXZ ... starts with '?') ... then the driver crashes ... SERIOUSLY ?! what the hell happends under the hood - I'm very curious the code behind this string comparition....

(and no, I didn't used the original string pointer from reflection somehow, I'v copied it into managed string table, so the string pointer was valid (at first when I discovered the solution, that was my first thought that the string pointer was somehow broken))

Ok back to assignments - what is the point of assigning individual shaders of hit group to LocalRootSignature ?

From what I'v read all shaders in the HitGroup need to share that same LocalRootSignature - right ?

(or is that some of the shaders in hitgroup could have LocalRootSignature, and others don't ? - it's not crystal clear to me)

As for assigning raytracing config to HitShader(s), MissShader(s) and all HitGroups, without this step, the code runs fine on NV2080, but does not produce ID3D12StateObject if ran on NV1080 with some internal error (again) (both of them on latest NV drivers)

And I'm not assigning raytracing pipeline config to anything ...