MSAA is not trivially supported for deferred rendering as is the case for forward rendering.

I do not want multi sampling in my deferred parts. On the other hand, I still want to use multi sampling in my forward parts. Below, the deferred -> forward connection is problematic since the depth buffer for deferred is a TEXTURE2D and for forward is a TEXTURE2DMS, and both depth buffers are not independent. Furthermore, I want AA for all models in the end (not only the cheap emissive and transparent models).

GBuffer packing (opaque_models) -> GBuffer unpacking (opaque_models) -> Forward (emissive_models) -> Forward(transparent_models) -> Forward (sprites)

So what if SV_COVERAGE is stored as well in the GBuffer packing phase? If I output the stored SV_COVERAGE in the GBuffer unpacking phase instead of the generated one (for the quad) and use a multi-sampling RTV, I have deferred MSAA right? Or am I missing something?

Furthermore, I could output my own SV_DEPTH as well to solve the TEXTURE2D <> TEXTURE2DMS problem?

It does not work like that, msaa rely on supersamplingthe depth, but you execute yourpixel shader only once. You could get a triangle writing a full coverage pixel, then one writing only half.

You can always copy fragments from and to ms textures, probably at the cost of depth decompression.

Most deferred engine use post process aa to not pay the memory craziness and bandwidth sadly

10 hours ago, galop1n said:

It does not work like that, msaa rely on supersamplingthe depth, but you execute yourpixel shader only once. You could get a triangle writing a full coverage pixel, then one writing only half.

You can always copy fragments from and to ms textures, probably at the cost of depth decompression.

 

Most deferred engine use post process aa to not pay the memory craziness and bandwidth sadly

 

So the only problem is the depth? Instead of one depth value, you will have at most 32 not-accessible depth values? But in normal MSAA, PS is only evaluated once per pixel. For which depth is this?

The coverage part can be mimicked, although at a wrong depth. What if you render a quad for the lighting first and store the colors. Next, you execute the VS of the models, fetch the color in PS while using MSAA? Now you should have MSAA? (Unfortunately you have no early z-testing since your depth buffer is initially empty).

This seems more computationally expensive, but you do not need to mark and heuristically fix edge pixels, nor do you need post-process AA.

19 hours ago, matt77hias said:

I do not want multi sampling in my deferred parts.

So: you render deferred to an MSAA target, but use SV_COVERAGE to say "this covers every sample" and now you can do transparent to the same MSAA target afterwards. A few problems:

1. Custom SV_COVERAGE outputs can only reduce the coverage mask to less than what the rasterizer produced. If the rasterizer says you only covered 1/4th of the pixel, you can't use SV_COVERAGE to force your PS output to be written to every pixel (without getting into vendor-specific techniques, at least).
2. If deferred isn't going to make use of MSAA, then using an MSAA G-buffer is just multiplying your memory usage by 2/4/8 for no reason  (the lighting accumulation buffer should be MSAA, but G-buffer ideally wouldn't be).

If you want no-MSAA on deferred and then MSAA on transparents later, you're probably forced to manually copy your non-MSAA opaque depth buffer into a second MSAA depth buffer before the transparency pass

If you want MSAA on deferred, the straightforward way is slow. In your deferred lighting shader, fetch every sub-sample and do lighting individually on each of them. 4xMSAA then requires 4x the lighting math ;(

The fast way that I know of is -- In your deferred lighting shader (which you've written as a compute shader ), just perform lighting for sub-sample #0 out of the gbuffer, as this will be correct for 95% of pixels (the ones that aren't edges). Also use an edge detection filter (depth discontinuity or normal discontinuity, and SV_COVERAGE wasn't 100%) to collect a list of pixels that require full shading of every sample. Then run your deferred lighting shader again, this time processing pixel coordinates out of that list instead of every pixel on the screen.

4 hours ago, Hodgman said:

The fast way that I know of is -- In your deferred lighting shader (which you've written as a compute shader ), just perform lighting for sub-sample #0 out of the gbuffer, as this will be correct for 95% of pixels (the ones that aren't edges).

And what about re-running the VS on the models and fetching a color in the PS while using MSAA?

1. GBuffer packing (opaque_models): write to multiple non-MS RTVs (GBuffer)
2. GBuffer unpacking (opaque_models): write to one non-MS RTV (Color Buffer)
3. Forward (opaque_models): fetch color from the Color Buffer and write to one MS RTV (back buffer)

Pros: Deferred MSAA and alpha-to-coverage transparency

Cons: VS invocation/vertex + texture load/fragment

4 hours ago, Hodgman said:

Custom SV_COVERAGE outputs can only reduce the coverage mask to less than what the rasterizer produced. If the rasterizer says you only covered 1/4th of the pixel, you can't use SV_COVERAGE to force your PS output to be written to every pixel (without getting into vendor-specific techniques, at least).

This is not a problem. The screen quad will always result in full coverage.

5 hours ago, matt77hias said:

This is not a problem. The screen quad will always result in full coverage.

I was thinking of the g-buffer pass, which populates the MSAA depth buffer with opaque's depth values. It would be nice to do non-MSAA rendering into an MSAA depth buffer here, allowing it to be used in later MSAA passes, but I don't know if that's possible. 

1 hour ago, Hodgman said:

I was thinking of the g-buffer pass, which populates the MSAA depth buffer with opaque's depth values. It would be nice to do non-MSAA rendering into an MSAA depth buffer here, allowing it to be used in later MSAA passes, but I don't know if that's possible. 

In theory yes, this is the quality parameter at resource creation, you could in theory have a MSAA 4x depth buffer with 4 fragments, and MSAA 4x color targets with only one fragment, the GPU then have some rules to keep/discard the fragments that could not fit. We can do that on ps4 and xbox one and write fancy resolves to try to rebuild/guess the missing fragments ( looking at neighbors, etc... ).

But i doubt D3D on PC will give you enough control to know what would happen behind the hood, and it is a GCN knowledge, on nVidia, who knows.