Thanks guys, I will thoroughly experiment with different methods and GPUs. I will be conducting benchmarks on the Sponza scene with Nvidia GTX 1070, GTX 960, AMD RX 470 and Snapdragon 808 GPUs. All of them will be timed for Input layout rendering method, Custom fetch with typed buffers and custom fetch with raw buffers.

I will post results in this topic when I am finished.

A Quick spoiler: AMD RX470 is performing surprisingly well even with typed buffers custom fetch (screw that it has virtually no difference whatsoever), while GTX 960 suffers greatly with doubled rendering time.

I have done the testing for an AMD and an NVIDIA GPU, the Snapdragon 808 will have to wait as setting up the scene for that will take some more time. I will also post the results for the GTX 1070 later.

Here you go:

I have attached a txt file with easier readability.

This is quite painful for me because I wanted to implement some features which require the custom fetching but seeing that it works so slow on nvidia it seems like wasted effort.

By the way, to quickly implement this, I bound my vertex buffers to texture slot 30 and upper, could it matter in performance?

Side note: It seems that this way the CPU time is also higher because VSSetShaderResources takes a longer time than IASetVertexBuffers. :(

Are you using 1 buffer for all attributes or 1 per attribute ?
If you're not using 1 for all attributes you should probably try that.

I am using 1 buffer per attribute (SoA). I recently switched from AoS layout to SoA layout and a complete rewrite of the scene rendering pipeline to allow more flexible buffer binding and more cache efficiency in depth only passes of which there are a lot more than regular passes. This gained me a substantial performance boost on both AMD and Nvidia, I am not interested in going back.

I meant 1 buffer and all Positions, then all Normals and so on...

So still SoA but in a single buffer, each attribute array appended to the next if you will.

In general, the reason for different types of seemingly similar resources is that at least one major IHV has (potentially legacy) fast-path hardware that differentiates between them. There are a number of buffer types which perform differently on NV GPUs while AMD's GCN GPUs simply don't care. You're seeing hardware design issues leaking through the software abstractions.

Ideally, we would just have buffers read by shaders and nothing else, not even textures. (I mean come on, texture buffers?) GPUs haven't reached that level of generalized functionality yet. MS originally pitched this design when they were sketching out D3D 10 and of course the IHVs explained it wasn't feasible.

I meant 1 buffer and all Positions, then all Normals and so on...

So still SoA but in a single buffer, each attribute array appended to the next if you will.

Why do you think it would be better? I imagine it would be harder to manage, because now you would even had to provide the length of the buffers to know the correct offset in the shaders.

Why do you think it would be better?

On 2017/6/1 at 11:59 PM, turanszkij said:

I am using 1 buffer per attribute (SoA). I recently switched from AoS layout to SoA layout and a complete rewrite of the scene rendering pipeline to allow more flexible buffer binding and more cache efficiency in depth only passes of which there are a lot more than regular passes. This gained me a substantial performance boost on both AMD and Nvidia, I am not interested in going back.

I did some tests on mobile GPU, all tests use AOS style buffer.

Adreno 418, SSBO is slower (50fps) than Vertex Buffer (55fps),

Adreno 512, no performance difference between SSBO and Vertex Buffer, this might because Adreno 512 has a unified memory model.

Mali(T7xx, T8xx, G71), none of those GPU support SSBO in vertex shader(although they support opengl es 3.1). 

GL_MAX_TEXTURE_BUFFER_SIZE is about 64KB on Mali, so I can't use texture buffer for vertex pulling either.