Advertisement

How to learn from Quake source code

Started by November 05, 2017 01:16 AM
46 comments, last by h8CplusplusGuru 7 years ago

Want to learn C++ to but.

Problem I have with these old engines are.

its legacy C & C++ 

Modern C++ gets Functional features.

The C++ I would go for is at least from C++14 to 17

OpenGL has modern replacement Vulkan.

And those very old engine are made for old single core Hardware. Modern CPU can do a lot of threads.

Why is John Carmack so interrested in Functional programming?

On 11/4/2017 at 9:36 PM, Michael Davidog said:

I don't know c++ very well yet

I hate to just pile on, but if your understanding of the language isn't great, then looking at the code will do you no good.  Understanding a language, understanding game dev or software dev techniques, and understanding an existing code base are all different and separate challenges.  If you want to learn the language, then learn the language first.  THEN start diving into understanding how existing stuff works.

I think something worth mentioning that hasn't been explicitly said yet is that (large) games are rarely made by one person.  That means that there's no one person who knows how every piece of a game works.  The person who does the physics might not know how the rendering works.  Someone who did all the shaders might know nothing about the code that takes player input.  In my experience, there's almost never one person who has a deep knowledge of the ENTIRE thing - and it's not very realistic to try to do that.  That's not even counting things like middleware or libraries that the game uses, which again, the people working on the game probably don't have a super deep understanding of the inner workings of those libraries.   Learn the piece you want to learn, don't aim for the whole thing.

Advertisement

It's not just because it's legacy C or legacy C++. Quake and Doom source code aren't that perfect beauty that should be aimed to.

Some sources have a bunch of global variables spread everywhere, and I mean everywhere, not at the top of the source file, but in between functions definitions.

Some functions are huge monoliths, which is a bad practice, too.

Some piece of the content are just commented out piece of code, not even removed from the source, but just unused.

Just because the game is good, doesn't mean the source code behind it is.

1 hour ago, ferreiradaselva said:

Some functions are huge monoliths, which is a bad practice, too.

IIRC Carmacks valid argument here is: If piece of code is used only once and in one place, there is no need to clutter it to smaller functions - it becomes just more likely to introduce bugs and harder to read.

Keeping out commented code is no bad practice either. Better than just leaving it in while unused, better than rewriting it again if it turns out you need it, and it reminds yourself of what have been issues or alternatives. Probably they had better things to do than removing this before they released it to the public so we can learn.

Declaring global variables near the place where they are being used makes sense too. Using them is not always bad practice, there is a reason they exist. Some people even use the goto statemant in a good way.

And finally, code is not good because it looks beautiful or modern. It is good if it works, is efficient and the number of bugs is small enough. That's the case here. It's also readable. 

44 minutes ago, JoeJ said:

IIRC Carmacks valid argument here is: If piece of code is used only once and in one place, there is no need to clutter it to smaller functions - it becomes just more likely to introduce bugs and harder to read.

Mr Carmack is VERY much aware of what he is doing and why. He is also very much aware of the precise conditions when to do this and (at least as important) when NOT to do this.

Your average new programmer here is very much unaware of pretty much everything. So while there are certainly cases where such practices make sense, you don't want to expose new programmers to it, until they have some understanding of the advantages and disadvantages of the various approaches, and when it is safe to break the general rules.

Give them time to learn writing decent general purpose code for the first 90% of the times.

 

44 minutes ago, JoeJ said:

It's also readable.

Highly optimized code is not readable unless you're very deep into the matter. In this case, you additionally need to understand computers of the previous millennium, which had very different problems and problem areas than computers nowadays. While these things are interesting, it's not helping a new programmer to write generally proper code, which is the aim of this forum.

As such, while reading quake source code definitely interesting, it's not a very efficient way to learn programming in the general sense.

2 hours ago, JoeJ said:

IIRC Carmacks valid argument here is: If piece of code is used only once and in one place, there is no need to clutter it to smaller functions - it becomes just more likely to introduce bugs and harder to read.

"Harder to read" is quite subjective.

Having worked on code written this way, and having refactored said code to use multiple smaller functions, I find it much easier to follow the code blocks when they were shorter. It is claimed that large, monolithic functions are better because they don't introduce unnecessary boilerplate, which is true - but this ignores the fact that the "boilerplate" can be hugely useful when actually navigating the code after the fact - which is important because on large legacy projects we tend to spend more time navigating code than actually writing it. The cost of writing the boilerplate is negligible over the lifetime of the project and in the case of simply breaking a function out into multiple functions, does not generally contribute to code bloat if it is done at the proper granularity level, unlike certain other kinds of boilerplate that do little more than add layers of abstraction. Breaking a monolithic function into smaller functions is a transformation that adds value in the long term.

Modern languages also allow us to make "local functions" - in C++, through the use of local lambdas with captures. One can contain any "clutter" that results from breaking a monolithic function down into sub-functions within the monolithic function itself, rather than polluting the local namespace with external functions, so that particular argument (which I know wasn't mentioned here, but I've heard it argued) doesn't really need to apply anymore.

2 hours ago, JoeJ said:

Keeping out commented code is no bad practice either. Better than just leaving it in while unused, better than rewriting it again if it turns out you need it, and it reminds yourself of what have been issues or alternatives. Probably they had better things to do than removing this before they released it to the public so we can learn.

I fundamentally disagree. Commented out code is, at best, an inferior way to duplicate the functionality of your version control system, and outright misleading at worst. Pray tell, do you actually maintain your commented out code through refactorings and bug fixes, "just in case"? If you really need that code back, you can just go back through your version control's commit/checkin history for the file that contained it. In my experience, chances are good that if you need the same functionality again later, the context will have changed sufficiently that the original code would not work as written, anyway.

I'd also argue that leaving commented code around "just in case it's useful" could contribute to perpetuating a culture of copy-and-paste programming, which I feel most of us can agree is not a culture conducive to producing maintainable code.

2 hours ago, JoeJ said:

And finally, code is not good because it looks beautiful or modern. It is good if it works, is efficient and the number of bugs is small enough. That's the case here. It's also readable. 

"Good code" has come to mean all of what you've said plus "code that can be maintained long term without your successors wanting to shoot you in the face." Does the Quake engine meet that specification?

If your code works, is efficient, and bug-free, but difficult to navigate and modify without breaking stuff, you have not written "good" code. You have at best written "throwaway code that gets the job done." In a world where the industry is moving towards "games as a service" and where at the very least, most major games get patched, extra content, or even outright sequels using the same codebase, I would say that throwaway code of any sort is no longer acceptable. I confess I've been forced to write throwaway code to solve a problem at the last minute like I imagine most programmers have, but I felt really dirty doing it and was constantly aware of how terrible it was that I was doing it.

edit: I was going to say something on the subject of global state and how it can hurt you in an increasingly parallel world, but others have made that argument in the past and I don't think we need to rehash it for the nth time. Having maintained large C++ codebases where global state was the standard way of doing things, even in a single threaded environment they've caused me more pain than I felt they were worth.

Advertisement
13 minutes ago, Oberon_Command said:

Does the Quake engine meet that specification?

Honestly i don't know - I've only skimmed sections of code that were interesting to me when i wrote a similar engine. The code looked good to me. It has been used in many games, so i assume it was maintainable well enough. Probably it could have been designed to make it easier creating a game different from an FPS like modern engines allow us to, but is this a matter of code quality? Or just a natural progress that happened with time?

However - i'm not going to argue with you guys about such things. I agree with most of your points - some are subjective.

Found the source in case i quoted Carmack inaccurate: http://number-none.com/blow/john_carmack_on_inlined_code.html

 

51 minutes ago, JoeJ said:

Found the source in case i quoted Carmack inaccurate: http://number-none.com/blow/john_carmack_on_inlined_code.html

I've read that post before and I stand by my points counter to the practice of excessive inlining on the grounds that it doesn't scale all that well and can obscure where things happen when navigating the code. It's worth noting that this post is from 2007, with an addendum from 2014 - quite some time after Quake shipped and still some time before C++'11 and C++'14 went mainstream. Quite a few of the points he makes actually have nothing to do with code inlining per se - they're more to do with architectural choices. The example he used was an aerospace thing, as well, and I'm not confident that what works for aerospace would work all that well for games.

For perspective, a few months ago I spotted a function that was multiple thousands of lines long that went something like this:


void foo()
{
  // 500 lines
  
  switch (thing)
  {
    case a:
      // 30 lines follow
      break;
    case b:
      // 1000 lines follow
      break;
    case c:
      // 300 lines
      break;
    case d:
      // 10 lines
      break;
    default:
      // 150 lines
      break;
  }
  
  // 30 lines
}

I ended up extracting each of the case blocks into their own functions because following what was happening when was proving too difficult, even though it had little to do with my current task at hand. Carmack's email on inlining would seem to agree that this is a good idea, actually:

Enclosing pages of code inside a conditional or loop statement does have readability and awareness drawbacks, so leaving that code in a separate function may still be justified

5 hours ago, JoeJ said:

IIRC Carmacks valid argument here is: If piece of code is used only once and in one place, there is no need to clutter it to smaller functions - it becomes just more likely to introduce bugs and harder to read.

Debatable and that's why inline functions exists. Even when OK, not the kind of approach to be passed to a learner, as @Alberth said:

5 hours ago, Alberth said:

Mr Carmack is VERY much aware of what he is doing and why. He is also very much aware of the precise conditions when to do this and (at least as important) when NOT to do this.

Your average new programmer here is very much unaware of pretty much everything. So while there are certainly cases where such practices make sense, you don't want to expose new programmers to it, until they have some understanding of the advantages and disadvantages of the various approaches, and when it is safe to break the general rules.

 

5 hours ago, JoeJ said:

Keeping out commented code is no bad practice either. Better than just leaving it in while unused, better than rewriting it again if it turns out you need it, and it reminds yourself of what have been issues or alternatives. Probably they had better things to do than removing this before they released it to the public so we can learn.

In addition to what @Oberon_Command, we have good version control software for that task. And, tbh, I doubt "not rewriting it again" or "reminds yourself of what have been issues or alternatives" are the only reasons here. Corporate code is different from the code that one can take the time to write, with all patience as a hobbyist. Ideally, learn as a hobbyist is better, write the code with all the time you have, no deadlines to worry about. Corporate programming must worry about delivery and deadlines, and it does affect code quality. It's not that people working on companies are bad programmers (not saying that), it's that they know when to prioritize time over good practices. A learner should prioritize good practices until they are secure enough to break their rules under certain circumstances (shipping delay).

5 hours ago, JoeJ said:

Declaring global variables near the place where they are being used makes sense too. Using them is not always bad practice, there is a reason they exist. Some people even use the goto statemant in a good way.

Yes, I use global variables, too, and the goto, too. But, by having global variables in the middle of the code:

  • will make difficult to make something multithreaded (you will have a harder time locating where are the globals affected my a threaded function)
  • will cause needless differences in the versioning when you have to move those variables from one point to another (which happens when a new function needs to use that variable, too, and could be avoided if they were on top)

To let it be clear: I'm not shitting on Carmack (or his team) work, but I can't help to feel bothered when I see the DOOM/Quake code being pushed as holy to beginners. And it's not.

7 hours ago, Alberth said:

Mr Carmack is VERY much aware of what he is doing and why. He is also very much aware of the precise conditions when to do this and (at least as important) when NOT to do this.

He might be in that state now. By his own admission (can't find the source at the moment, but probably one of the interviews Fabien links to in the Quake blogs, or his .plan archive, or Masters of Doom), he wasn't really sure what he was doing a lot of the time back when he was writing these early engines; he was muddling through as best he could, and learning things left, right and center along the way.

The Quake 1 engine is probably the most approachable of the open-sourced IdTech engines - I don't have it pulled down at the moment, but I was pretty deep into looking at it a few years ago, and I want to think it was on the order of 50k lines of fairly standard C89 code, which is more manageable to keep in your head than the later, larger engines. And it doesn't have a ton of the inline assembly and DOS-hacks of Wolfenstein or Doom.  The OpenGL rendering pipeline is archaic, but its pretty standard for the way you had to do things back in that OpenGL 1.0 era.

That being said, exploring the code base is more of an archaeological expedition than anything that you would want to take as an example of best-practices.  The hardware available now is very different than what was available then, and we've got much more powerful tools, as well as twenty-odd years of insights into better ways to structure game engines.

Eric Richards

SlimDX tutorials - http://www.richardssoftware.net/

Twitter - @EricRichards22

This topic is closed to new replies.

Advertisement