47 minutes ago, Krypt0n said:

If you'd share your RD capture, I'd take a look. (no promises I find it, tho! )

No worries. I'm already grateful you're willing to take a look :-) Here it is double_patch.rdc (captured with v1.0).

I've checked the output mesh, it seems to be watertight. The Pixelshader also seems to output the proper values. (either pure red, with bg==0 or r==g==b).

watching every individual sample-layer, there are intermediate colors between pure red and pure gray, if that was a rasterization artifact (e.g. due to precision issues), you'd have red or gray, nothing inbetween. This mixing of colors still make me strongly believe CSAA is running. (Maybe the driver ignores your settings or something?)

Have you the same issue with 4xAA? could you ramp up the quality as far as it goes and check the output?

I'm a bit out of ideas. sorry

19 hours ago, Krypt0n said:

This mixing of colors still make me strongly believe CSAA is running. (Maybe the driver ignores your settings or something?)

Sounds reasonable, but hard to check 

19 hours ago, Krypt0n said:

Have you the same issue with 4xAA? could you ramp up the quality as far as it goes and check the output?

Yes. 4x and 2x are equally bad. Increasing Quality (at 8 samples, 32 is max on my gpu) actually reduces - but doesn't fix - the problem. I have been sticking to Quality == 0 because that makes it easier to implement. If you want to use the maximum quality level, in theory you'd need to check the minimal maximum quality level for each group of render targets used at the same time. Which is quite some bookkeeping. And if I understood correctly, a higher quality level does not necessarily mean better AA. So I thought: let's just stick to level 0 - which is always valid - and assume the manufacturers have put a decent default at that level

19 hours ago, Krypt0n said:

I'm a bit out of ideas. sorry

Me too  But thanks again for all help and suggestions. I guess I'll just assume this is a gpu/driver issue. I hope once I switch to (vertex cache optimized) indexed rendering, the problem will disappear mostly anyway...

Just throwing one more idea in - assuming you have the most recent drivers, can you try installing older ones? It is possible that NVidia made some mistake in most recent drivers (in which case reporting it to NVidia is always viable option), it wouldn't be first time a bug was introduced.

I downgraded to quadro driver version 385.90 (from November 2017). Didn't help. Oh well, it was worth a try 

On 8.3.2018 at 5:34 PM, Koen said:

I downgraded to quadro driver version 385.90 (from November 2017). Didn't help. Oh well, it was worth a try 

Hey,

i just registered to let you know how I fixed this. I had exactly the same issues which looked like this:

https://imgur.com/a/sOoa7

It took me about 8hours of debuggung. I found out that the PS Invocation Count is changing from frame to frame by +-50000! Then i changed SV_Coverage to see individual samples and saw that the MSAA samples itself were fine and I knew it had to be a state issue.

So after some playing around I found that if I set

MultisampleEnable = FALSE;

inside the D3D11_RASTERIZER_DESC then Multisampling will still work but there wont be any flickering and holes between polygons!

I hope this will help everyone with the same issue!

regards
Valdiralita

2 hours ago, Valdiralita said:

I hope this will help everyone with the same issue!

My hero! It sure did fix the issue.

2 hours ago, Valdiralita said:

I found out that the PS Invocation Count is changing from frame to frame by +-50000! Then i changed SV_Coverage to see individual samples and saw that the MSAA samples itself were fine and I knew it had to be a state issue.

I don't see how you came to that conclusion, though  Or is it just: tried everything else, so it must be a pipeline state?

Also, if anyone happens to have an explanation for this interesting phenomenon where you have to disable msaa to get proper msaa  I'd be happy to hear.

Thanks again for all the help!

5 minutes ago, Koen said:

My hero! It sure did fix the issue.

I don't see how you came to that conclusion, though  Or is it just: tried everything else, so it must be a pipeline state?

Also, if anyone happens to have an explanation for this interesting phenomenon where you have to disable msaa to get proper msaa  I'd be happy to hear.

Thanks again for all the help!

Yeah, I pretty much checked EVERYTHING else so it had to be a pipeline state.

Glad that it fixed it for you too. My best guess is that the MultisampleEnable bool is for an automated multisampling and messes up with the manually set sample count.

Oh just another info: This change doesnt work well with line rendering and setting IsAntialiasedLineEnabled to true produces this:

I ended up creating another RasterizerState that i use for line rendering which uses IsMultisampleEnabled = true and produces the desired result:

regards

4 minutes ago, Valdiralita said:

Oh just another info: This change doesnt work well with line rendering

Yup, I already noticed  My api/gpu abstraction uses a PSO approach with almost all state grouped together, so I can easily detect a request for anti-aliased lines and triangle edges, and create a different rasterizer state for those cases.

I still have to think about how to properly handle tessellation and geometry shaders, though. For now I'll just assume they always produce solid triangles