I've run into a puzzling rendering issue where triangles in the back, bleed through triangles in front of them. This screenshot shows the issue:

The leftmost picture is drawn with no msaa and shows the expected result. The middle picture is exactly the same, except with msaa enabled. Notice the red pixels bleeding through at some triangle edges. The right picture shows a slightly rotated view, revealing the red surface in the back. In the rotated view, the artefact goes away, apparently because the triangles are no longer directly facing the camera.

The issue only occurs on certain gpus (as far as I am currently aware, nvidia quadro K600 and intel integrated chips).
When using a WARP device or the D3d11 reference rasterizer, the problem does not occur.

The triangle mesh also affects the result. The two surfaces in the screenshot are pieces of a skull, triangulated from a 3d image scan using some form of marching cubes. So the triangles are in a very specific order. You can also see this in the following RenderDoc pixel history:

RenderDoc shows the 'broken' pixel covered by two triangles -- I'd actually expect at least three: two adjacing the edge, and at least one from the back surface. The two triangles affecting the final pixel color are consecutive primitives. The front one has a shader depth output of 0.42414, the back one has a depth of 0.73829, but still ends up affecting the final color.
If I change the order of the triangles -- for instance by splitting the surfaces and rendering each surface with its own draw call -- the problem also goes away.

I understand that msaa changes rasterization, but shouldn't adjacing triangles still be completely without gaps? All sample positions within a pixel should be covered by the triangles on both sides of the edge, so no background should bleed through, right?
For the record: I did check that there are no actual gaps/cracks in the mesh.

Is this a driver/gpu bug? Am I misunderstanding the rasterization rules? Thanks!

This looks weird at least. If you look at the output of RenderDoc (although I haven't worked with it - can it properly show values per-sample for given pixel (as with MSAA you might have multiple rasterization results per pixel) - as when I work with ray tracers, you mostly have a way to examine results per sample, and not per pixel - in case of super-sampling).

In the Tex After - why is Depth value equal to '?' after Primitive 2276 (which means that value in Depth buffer is undefined at that point, I assume).

And same goes for Depth value for primitive 2277 - if it did pass depth test, shouldn't that value be written in the output?

I'd check what is with your depth testing. Especially this - do you have correct sample count/sample quality set up for the depth buffer?

You might be running MSAA with less color than coverage samples, that's called CSAA http://www.nvidia.de/object/coverage-sampled-aa.html

It can lead to these kind of artifacts, simply because the color values might be averaged (as you have to fit e.g. 16 output colors into 4 colors saved colors).

You can change the color and sample count, but that depends on your API. (Check the link for various ways. DX11 and DX12 are very similar to DX10 in that regards).

If you zoom in on the mesh output view, looking at the wireframe, is it actually closed? It could be that there are actually tiny cracks in your mesh that aren't visible at the normal resolution? 

Precision (t-junction) issues might be at fault. Is there a debug view showing the triangles to see if the cracks line up? Basically the internal mesh rendering of these specific GPUs could be throwing whatever precision it thinks is best at your triangles, altering them slightly due to quantization errors and allowing tiny gaps to bleed through.

Thanks for all your suggestions.

17 hours ago, Vilem Otte said:

I'd check what is with your depth testing. Especially this - do you have correct sample count/sample quality set up for the depth buffer?

Count is always 8, quality is always 0. Coincidentally there was a new RenderDoc release yesterday, and it allows viewing all samples separately (maybe this was already in before, but I hadn't seen it). This is all the samples for one of the offending pixels (it's a wide image - I hope it doesn't mess up layout too much):

So according to RenderDoc, the depth values are actually ok, but for some reason the latest primitive is always used, regardless of depth. Depth-stencil and blend states all seem fine: depth test less or equal, depth write enabled, blending and stencil test disabled.

14 hours ago, Krypt0n said:

You might be running MSAA with less color than coverage samples, that's called CSAA http://www.nvidia.de/object/coverage-sampled-aa.html

This doesn't seem to be the problem. It is disabled in the nvidia control panel, and I create my render targets with a sample count of 8, and a quality  level of 0. It could explain the weird sample results seen in the picture above, though...

10 hours ago, Hodgman said:

It could be that there are actually tiny cracks in your mesh that aren't visible at the normal resolution? 

5 hours ago, FreneticPonE said:

Precision (t-junction) issues might be at fault.

I explicitly checked this on cpu side, and in RenderDoc: the triangles use identical vertices, and there are no t-sections. Also, zooming in makes the problem go away instead of making it worse :-)

Running through the code once more, I noticed my depth buffer has the D3D11_BIND_SHADER_RESOURCE flag set (because I 'manually' resolve the depth buffer to non-msaa for dual-depth-peeling transparent objects), which could maybe be a shady practice :-) but removing the flag didn't solve the issue. For the rest, all resources and views seem ok:

Color render target D3D11_TEXTURE2D_DESC
  Width     500                       
  Height    417                       
  MipLevels 1                         
  ArraySize 1                         
  Format    DXGI_FORMAT_B8G8R8A8_UNORM
  SampleDesc DXGI_SAMPLE_DESC()
    Count   8
    Quality 0
  Usage          D3D11_USAGE_DEFAULT     
  BindFlags      D3D11_BIND_RENDER_TARGET
  CPUAccessFlags 0                       
  MiscFlags      0

Render target view D3D11_RENDER_TARGET_VIEW_DESC
  Format        DXGI_FORMAT_B8G8R8A8_UNORM     
  ViewDimension D3D11_RTV_DIMENSION_TEXTURE2DMS
  Texture2DMS   D3D11_TEX2DMS_RTV()

Depth buffer D3D11_TEXTURE2D_DESC
  Width      500                     
  Height     417                     
  MipLevels  1                       
  ArraySize  1                       
  Format     DXGI_FORMAT_R32_TYPELESS
  SampleDesc DXGI_SAMPLE_DESC()
    Count   8
    Quality 0
  Usage          D3D11_USAGE_DEFAULT                               
  BindFlags      D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_DEPTH_STENCIL
  CPUAccessFlags 0                                                 
  MiscFlags      0
  
Depth stencil view D3D11_DEPTH_STENCIL_VIEW_DESC
  Format        DXGI_FORMAT_D32_FLOAT          
  ViewDimension D3D11_DSV_DIMENSION_TEXTURE2DMS
  Flags         0                              
  Texture2DMS   D3D11_TEX2DMS_DSV()

1 hour ago, Koen said:

This doesn't seem to be the problem. It is disabled in the nvidia control panel, and I create my render targets with a sample count of 8, and a quality  level of 0. It could explain the weird sample results seen in the picture above, though...

 

Could you give "8" for "Quality" a try?

Sure! It doesn't fix the issue :-) I had already tried '1' before - also no improvement.

darn, I was so sure that would be it.

If you'd share your RD capture, I'd take a look. (no promises I find it, tho! )