Hello, everyone.I really need to share this and ask if anyone else has experienced this.It's not ENTIRELY programming related, so that's why I'm posting it in the lounge.So basically, I've rewritten from scratch my game engine 7 times and I can't afford to do it again.The problem is, that every time I start out with the "perfect design, perfect codestyle, perfect performance, small as possible and clean as possible code, etc", yet some 50k lines later I start to hate the engine, I get a "This Engine Sucks"(TES) moment.I look at my code and then I look at other examples/tutorials/open source engines and I idealize them, while hating on my own work.I've also done a lot of worthless refactors and then refactored back to the original idea.Now the dissonance is back after reading this:
http://software.intel.com/sites/default/files/m/d/4/1/d/8/Designing_a_Parallel_Game_Engine.pdf
My current engine supports multithreaded command list generation in DX11, but the system is specific to the renderer and it's basically just thin wrappers over windows handles and a couple of windows threading-related functions.So is my decompressor for the texture streaming(placed on a free CPU core), however after reading this I envision some god-like threading system, which allows me to split everything possible in the engine into Task objects.This however would require a massive restructuring in the entire framework or to just start from scratch again and I can't do that anymore.Not sure how to get rid of this dissonence, until the threading issue came up, everything was going perfect.Even if I do somehow recode everything again, another dissonance would just come up eventualy.How do you guys deal with this?Say you coded something really hard, but it turns out to be offsetted from the original architecture you had in mind, do you start over from scratch or what?Any input would be appreciated.

Canonical answer to almost the same question.

There is no perfect design, the grass will always be greener. There will always be some new sexy way to do it, if you haven't heard of it yet it is approaching the horizon. And just because it is new and sexy doesn't mean that it is better at everything, there will be a trade-off.

It sounds like you might be designing in a vacuum. Write the games, not engines. When you're writing a game, whether the engine has a "god-like threading system" with "Task objects" is less important than whether you can actually achieve what you need within the current constraints that your engine has (there will always be constraints). Concrete goals with obvious milestones cut down the tendency to over-engineer and to over-think some of these things.

If your goal is to have the sexiest, cutting edgiest engine around, you'll have to accept that this is a moving target.

The main thing to realise is that the engine you haven't written yet will also be flawed and irritating, as is the case with all software that actually exists as opposed to imaginary software in your head. You don't imagine the areas that your architecture makes more difficult, you don't imagine the performance problems, you certain don't imagine all the bugs. If you can accept this, you'll find the willpower to resist this temptation comes easier.

50k lines just to see the engine does not work?

I believe you should slow down and write less better code instead. I think I am at about 90k for my system. It has been rewritten only once.

Also, I don't know why you people care about performance optimization before having something that works. Gamedev is full of smart people, I cannot thank them enough for their technological insights. But some of them do a disservice to the community by believing state of the art is something we should have at all costs. It's not.

Thanks god the times have changed and now most of the community is geared towards a "make it work first" attitude.

So, you have this feeling your system does not work.

My suggestion is: don't care about that.

Here's a list of things that don't work in mine:

• Memory management is ankward at best.
• Scripting is actually so flexible it shoot me in the foot several times.
• The material system is nowhere as flexible as intended, albeit already more complex than I want it to be.
• When I roam those forums, I see every newbie with super-fancy shaders and multithreading I ask myself: where did all my effort go?
• It actually doesn't meet my performance target on 1st gen atom.

I don't really care about those things because the game works. Sort of.

Things I really care about are the show stoppers such as having a lighting subsystem that makes me want to run away screaming with its inflexibility. And most of all,

this new import filter which should have been out in January.

Those are the things which you should care. Feeling is a thing, but being sure of a problem is another.

maybe good, maybe bad, is when one is left to realize that, actually, 50kloc actually is pretty small for a codebase...

to really be able to do a lot of stuff, typically people will be looking at codebases a fair bit larger than this.

yes, "elegant" and "perfect" aren't really achievable goals sometimes.
"basically works" is much more reachable, as well as trying to keep options open and avoiding boxing oneself into a corner (though: also try to avoid premature generalization).


sometimes, what works is basically to throw out any sort of planning or up-front-design design, and instead just throw something together and fiddle with it (in an experimental sense), and see if or how well it might work, ... (then, depending on initial results, a person might decide whether to invest more time/effort into developing the idea further).

a large amount of what one does may turn out to suck, but at least if they find out early that it sucks, it can be a sort of damage control (vs spending a lot of time/effort implementing something only then to find out that it sucks...).

and maybe sometimes just "go with the flow" or "listen to what the code seems to be saying" (not really sure how to better describe it, just that sometimes it seems almost like code has its own "voice" and its own "will", but it expresses itself very indirectly, more like a sort of general sensation...).

well, and the power of making things up as one goes along.

...