Hello guys,

title says it, I want create an api agnostic rendering layer (like bgfx, but not trying to compete). 
My idea was to specify an interface IRenderLayer that defines the specific functions. Implementations will then override these functions and implement api specific behavior.
However I always read that virtual function calls should be avoided in hot code paths and I also understand why. 
But then again, how should I implement runtime api selection without using runtime polymorphism? 

Thanks

I wouldn't worry too much about the use of runtime polymorphism. It's way down on the list of interesting or useful problems. 

The key point you need to think about is, what level will the abstraction exist on? There are two equally valid ways to do this. You can create "API-level" abstractions for buffers, textures, shaders, etc and implement common interfaces for the different APIs. Or you can define high level concepts - mesh objects, materials, scene layouts, properties - and let each API implementation make its own decisions about how to translate those things into actual GPU primitives and how to render them. There are a lot of trade-offs inherent in both approaches.

Thanks for the answer.
I want to implement the first option: API-Level abstraction and specify common interfaces for all APIs I want to target
But how do you efficiently implement different backends? 

The cost of virtual functions are usually greatly exaggerated in many posts on the subject.  That is not to say they are free but assuming they are evil is simply short sighted.  Basically you should only concern yourself with the overhead if you think the function in question is going to be called >10000 times a frame for instance.  An example, say I have the two API calls:

"virtual void AddVertex(Vector3& v);" & "virtual void AddVertices(Vector<Vector3>& vs);"

If you add 100000 vertices with the first call the overhead of the indirection and lack of inlining is going to kill your performance.  On the other hand, if you fill the vector with the vertices (where the addition is able to be inlined and optimized by the compiler) and then use the second call, there is very little overhead to be concerned with.

So, given that the 3D API's do not supply individual vertex get/set functions anymore and everything is in bulk containers such as vertex buffers and index buffers, there is almost nothing to be worried about regarding usage of virtual functions.  My API wrapper around DX12, Vulkan & Metal is behind a bunch of pure virtual interfaces and performance does not change when I compile the DX12 lib statically and remove the interface layer.  As such, I'm fairly confident that you should have no problems unless you do something silly like the above example.

Just keep in mind there are many caveats involved in this due to CPU variations, memory speed, cache hit/miss based on usage patterns etc and the only true way to get numbers is to profile something working.  I would consider my comments as rule of thumb safety in most cases though.

Last January I was in the same situation as you, and since then I have spent a lot of time writing an abstraction layer on top of DX11, DX12 and Vulkan. Here are some tips, however, I am by no means an expert so take everything I say with a grain of salt.

• Just go for the pure virtual interface, if that is what you are comfortable with. As others have stated, the performance does not need to be a problem if you think about how often you call the backend through the API.
• Decide early if you want to support new, lower level, APIs such as DX12, Vulkan or Metal. And if you do, develop first and foremost for those. It is a lot easier to write a backend using DX11 or OpenGL for a low level API, than writing a DX12 or Vulkan backend for a high level API. Certain things, such as fences, can just be skipped when writing a DX11 backend. But good luck trying to write a backend for DX12 when the API has no concept of a fence.
• Even if you don't plan on supporting low level APIs right now, think about bringing some of their concepts into your API, such as PSOs and command buffers. At least I much prefer packaging state together into PSOs rather than setting state all over the codebase. They tend to make the app code cleaner, and might even help you do optimizations such as hashing state and avoiding doing unnecessary backend calls.
• If you plan to support Vulkan, make render passes a first-class citizen of the API. They are the only way to render in Vulkan, and they are easy to emulate in the other APIs.
• Make a unified shader system. When I create a PSO, I load a shader blob from disk and pass it to the renderer. The app programmer never needs to know which underlying shader language is used. The blob contains "language slots" for all supported backends. Eg. the blob might contain HLSL bytecode for DX11/12, SPIR-V for Vulkand and GLSL for OpenGL. The backend pulls the correct code out of the blob and the others are unused. The blob also contains uniform metadata, ie. which binding slot is used for the constant buffer with name "cbLighting" etc.
• Don't write shaders in multiple languages. Cross-compile HLSL into what you need. glslangvalidator supports HLSL out of the box. There are HLSL-to-GLSL cross compilers available etc. Eg. https://github.com/LukasBanana/XShaderCompiler seems promising.

These are some things I have found helpful. Hope it helps!

Thanks so much guys, this really helps me

- Alright I will go with pure virutal interface. Decision made.
- Originally I wanted to, but having to write 800 loc just to draw a triangle was a turn off. So instead I will target AZDO opengl 4.5. Hope it's  somehow compatible with the other concepts.
- Command Buffers are essential with AZDO I think (because of Indirect draw calls)
- Well maybe I will target Vulkan if it's somehow the superset of the other APIs... 

Thanks again

1 minute ago, Jan Haas said:

- Originally I wanted to, but having to write 800 loc just to draw a triangle was a turn off. So instead I will target AZDO opengl 4.5. Hope it's  somehow compatible with the other concepts.

If you want this to work, I would start the implementation with two APIs. AZDO GL 4.5 and DX11 would be sufficient. Trust me, this will expose a lot of blindspots that you miss even if you're pretty familiar with multiple APIs. Been there, done that, have the wide-ranging commits to atone for my oversights.

Do you think AZDO and DX 11 are compatible? Because I feel the azdo style is completely different than dx11  

As well as picking between low-level (API wrapping) or high-level (scene type stuff -- models/materials/etc), you also have to choose whether you'll be making a state-machine API or a stateless API. GL/D3D/etc are all state-machines, so a simple wrapper will also be a state machine. Stateless rendering APIs are IMHO utterly superior though, so personally I'd recommend going down that path

3 hours ago, Jan Haas said:

But then again, how should I implement runtime api selection without using runtime polymorphism? 

The big alternative is to do compile-time API selection instead  

4 hours ago, Hodgman said:

The big alternative is to do compile-time API selection instead  

Can't on mobile, if you're looking for wide support. Is Metal available or run the GL ES pipeline? Is Vulkan available or run the GL ES pipeline?