So I've stopped writing journal entries on my project for a while. Didn't really feel like it anymore, after I took a hiatus for multiple months. Since then I've been working on multiple things, mostly cleaning up old messy parts of the engine. One of this things, that turned out something pretty cool in my mind, was the redesign of the rendering pipeline, that I wanted to share with you (can't promise I will keep up writing articles, but I'll try my best as long as it fulfills me). For now I'll split this into 3 articles that I'm going to write at once, with another 1 or 2 coming in the near future though.
What I mean by high-level renderer:
So just you know exactly what I'm talking about. By high-level renderer, or rendering pipeline, I mean the system in place that allows for rendering the world, post-processing effects etc... which pulls it all together. Say I have entities/models that I want to render, as well as some terrain, then some effects like HDR, SSAO, etc... all those in the correct order form a rendering pipeline, and the high-level renderer I'm going to present that I just designed allows to write all these things in isolation, and pull it all together so that you can see the fully renderer scene.
What I had to begin with:
The renderer originally was one of the first things I wrote, all the way back ~3 year ago. As you can imagine, it was one of the messiest parts of the engine, as knew very little compared to know.
To be perfectly honest, I didn't even have any high-level rendering framework/pipeline up until now. What I had was a low-level abstraction based up some of the information found in the classic frostbite rendering architecture post found in these forums:
https://www.gamedev.net/topic/604899-frostbite-rendering-architecture-question/
as well as some other forms of abstraction for primitives like textures, meshes, ... but on top of that, implementing specific rendering effects was a total free-for-all. There was some things that I kept, but others were just... well, you'll see in a minute.
The good:
Of the things I still kept to this day (in some form), there was a system of render queues and stages. A queue would take render-primitives with a sort-key, and execute those after the whole queue has been filled. Primitives are not directly assigned to the queue, but instead go through a stage first - the stage know which render targets to render the primitives too, and acts as part of the sort-key (so that whole stages are processed in order).
Thats a system I quite liked, it allows for deferred rendering of primitives, which is not only good for the sorting-part based on ie. depth and material, but also makes it easier to write code for rendering, since we do not have to think about the order in which primitives are drawn ourselfs.
The bad:
What wasn't that cool, at least after some time toying around with it, was how those queues and stages were set up. I used data-files for it, which is fine, but since I was using a plugin-based architecture, things quickly became messy. I ended up needing to implemented a "group" system on top of it all, so that passes could be inserted at the correct point.
This would usually mean I'd have on file for declaring the groups in the main-render plugin (since I wanted to support 3d and 2d):
And plugins would then add stages and queues to those groups, like so:// water plugin
Not only is it annoying to write per hand, and confusing as f*ck, but also prone to error and hard to debug. Things that were the most annoying are actually:
- Render targets had to be declared manually as assets, and referenced by name (later by asset UID, at this point it became even more confusing and nearly impossible to write by hand).
- There is no [s]easy[/s] way (no, just none) to allow multiple view with this. Just think of at minimum having the four view showing different sides of the scene you have in modeling-tools, but can also expect to being able to use in a game engine - just cannot be done. This is due to many reasons: the entire rendering process is part of the same thing (includind the editor-UI that finally outputs the rendered scene); render-targets being declared and used via assets; and many, many more - and you haven't even seen that code using that.
- Generating render-effects purely in code was horrendiously difficult. Not going into to much detail, but specially trying to write some render-code that is instantiated multiple times was a horror mainly due to the reference-by-name-thingy.
The ugly:
But the really bad stuff started when you tried to actually implemented a rendering pass. Oh boy. So aside from what I just mentioned above, there are no other systems in place. So I relied on some really fugly, dirty stuff - like abusing Entity-Systems for implementing things like an SSAO-render pass (mainly due to being able to pass messages). I'd usually grab the stages that were defined in the rendering data file, and then just feed it the rendering-primitives in an hijacked "update" function.
At first, handling things like camera constants was also a nightmare. I ended up just having a separate cbuffer for each stage, where I just put the camera data manually in all the systems that needed it. Later I moved to having a global cbuffer for that, and have the camera system put the constants in, but now the cbuffer was global to the renderer, which wasn't that nice eigther.
This was also another reason why I would have never been able to do stuff like have multiple scene views, or the like.
That was actually the point were I started to think about reworking this system - I've been working on allowing multiple views for the sake of preview etc... for some time, when I pretty much just had to rewrite the renderer in the process. I'm still very much pleased with what I came up with at the end, which ended up solving all the problems mentioned about so I'll show you that in the next article, which is right there:
