If you are the "Build from Scratch" type, I have a word of warning for you: Be very careful that you are not becoming a victim of the "Not Invented Here" syndrome. It's very common for engineers.

If you are debating whether to build an engine from scratch, I ask you to very carefully question your motives. It's 2015. There are free game engines (UE4, Unity) which can accelerate your game development process. Writing your own engine should mostly be an academic exercise to understand engine architecture. If you're using it as a means to create a game, you're probably doing it wrong -- unless you've got a very compelling business case for it.

I speak from experience on this... I wrote my own game engine and it took me a year. It was alright, it worked, but it wasn't professional grade and it was also a bit buggy. If I wanted to add a new capability to my game, I'd have to spend several weeks adding it into my engine. I finally got smart and used UE4, and I have not looked back.

The actual fundamental question is this: why is composability* so poor in our current languages/technologies?

I agree that this is a very interesting take on the problem. While I could speculate a lot on the actual "why"s of how our composability has regressed over the past decades, I don't think it'd wind up being productive :-)

What I'm personally more interested in is how we recapture the ideas of live-swapping code, of building systems that can talk to arbitrary other systems sans the stupidity of most of the "universal protocol" attempts that have been made in the past, and so on.


OK, fuck it, I'm gonna speculate a tiny bit. I think the proliferation of languages without clean ABIs (C++, cough) and the erosion of clean platform-level composablility features (like Unix espouses, and Windows doesn't) is a big part of the problem.

Here's why I say that: if you're running Smalltalk, you're an end-to-end Smalltalk system. You can compose arbitrarily because the whole system is self-contained and pieces you get from other sources use the same basic mechanisms to interact with each other. Same goes for the Unix philosophy of textual communication via piping; if the entire platform believes in the compositional abstraction, it works great. If it doesn't, you get PowerShell.

Contrast that with my current workstation: I run programs and scripts written in PowerShell, Python, Ruby, C, C++, C#, and probably a handful more. Aside from the C ABI, there's no clean way for all those to interact, and the C ABI is not first-class in many of those languages.


This leads to attempts at a lingua franca of code, which usually wind up looking like SOAP or something equally obnoxious.


So ever being the stubborn pragmatist: how do we move on from this situation?





Also, really good to see you again, Seyi!

Something I find is that once you start to want to go beyond the basics of an existing library, you end up having to develop such a deep understanding of the library that the balance of use this vs write your own becomes far less clear cut.

I struggled with the Bullet kinematic character controller for example. Not knocking a free open source library but the KCC that ships is awful and needs a lot of work.

Implementing my own did take some time, but I now not only have a solution that does what I want, I actually understand what every character on every line is for, since it's mine. So when I want to add features that do not fit easily into a physical simulation, it becomes far easier than grappling and hacking at Bullet.

Even the more battle tested KCCs in Havok or the other commercial engines will still give you problems if you are trying to do unexpected things with them, at which point the time you will need to invest in learning the library in order to customise it may become a larger hurdle than figuring it out for yourself.

Or maybe I am just a NIH in denial and rationalising :)

Or maybe I am just a NIH in denial and rationalising

I don't think that's it. The first time I invest time with a core mechanic or feature of a library, but then found weird, subtle bugs that were hard to find, and then couldn't fix it because the source wouldn't build, I begin to feel this way. After this happens time and again, I find myself looking for libraries, finding some that work, and then figure out what tech the are using under the hood. Then I just learn that and continue.

I'm doing this right now with Freetype for text. I tried using some libraries for text rendering, but they were all flawed in some way. So I learned to use the same library the libraries use. Turns out that Freetype is flawed to, and has some weird problems. But there isn't anything more low-level going on, so I'm using it. It isn't from scratch, but I tend to go as low level as possible without being ridiculous.

I'm a lot more careful in that context to use pre-existing tech where possible in order to offer maximum output and minimum risk in exchange for my pay.

You guys are lucky to be able to propose that at all in the first place.

One time I tried to use a formula for hashing (one line of code) taken from boost, and some mail came back saying I was using something from outside which is not acceptable.

In what industry to you guys work ?

I'm a lot more careful in that context to use pre-existing tech where possible in order to offer maximum output and minimum risk in exchange for my pay.

You guys are lucky to be able to propose that at all in the first place.
One time I tried to use a formula for hashing (one line of code) taken from boost, and some mail came back saying I was using something from outside which is not acceptable.
In what industry to you guys work ?

I do project management software for the offshore industry but ours is a tiny company so we have a lot of technical freedom.

I have heard of the attitude you describe. Must be a nightmare.

I'm a lot more careful in that context to use pre-existing tech where possible in order to offer maximum output and minimum risk in exchange for my pay.

You guys are lucky to be able to propose that at all in the first place.

One time I tried to use a formula for hashing (one line of code) taken from boost, and some mail came back saying I was using something from outside which is not acceptable.

In what industry to you guys work ?

Hah. That sounds familiar. One time my team was trying to fix buggy GPS code. It was taking this guy two whole weeks to fix it and it was still not fixed. I came to take a look at the code, and decided that the whole thing was stupid and dumb. So in 4 hours, I deleted the entire existing code and replaced it with mine. Not only I made it bug-free, it was 5x faster. QA confirmed and everyone was happy. Yay! Not.

The next day a massive drama took place among the management as soon as they heard me deleting an entire existing codebase. That was the end of my career at that company.

Moral of the story? Existing code does not always mean it's good. I have seen and maintained enough existing code bases that convinced me that existing code is almost always bad. Unless, they can wrap that whole ugliness in a black box and document it such that I never have to take a look at any of the innards. That's the only time I would ever consider reusing someone else's library, and that still has to go through my other filter: "Can I use this library in this and that way?" If that doesn't satisfy the requirement, I would drop that immediately.

Another story that I did the opposite was when I just joined this startup with no existing infrastructure, and I am planning of implementing a deployment strategy. Docker is all the rage now, and back then was still at its infancy stage, barely production ready. I spent my time evaluating Docker, creating test projects to see what it feels like to deploy stuff using Docker. After several days of playing around with it, I was actually able to use it the way I want it to. It wasn't perfect, but it was good enough.

However, that does not go without bumps and hiccups. I still found edge cases in which Docker didn't support that I had to hack it up with bash scripts here and there, but this is after I decided I want to use it.

But what does it really mean? From scratch.. what? Lots of you claim to do everything from scratch, but then mention OpenGL, DirectX or even SDL.

But what does it really mean? From scratch.. what? Lots of you claim to do everything from scratch, but then mention OpenGL, DirectX or even SDL.

To be honest an application or game developer can't really go lower level than directx or opengl without going to the driver level. There are so many different graphics cards out there now that if you tried go to hardware level you'd never get a game out the door, and if you did it would only run on a small percentage of pcs.

There's reinventing the wheel, then there's trying to reinvent the granite that wheel is carved from, as the shed discussion above has already explained...

But what does it really mean? From scratch.. what? Lots of you claim to do everything from scratch, but then mention OpenGL, DirectX or even SDL.

I never advocated everything has to be from scratch, but it tends to go that way the more I try to use others' tools.

But this is still from scratch anyway. You are just using OpenGL as a bridge to send your data and programs to the GPU. If there were not many different GPUs then it would be VERY profitable to communicate with it directly, though. That is a huge chunk of overhead.