Hi,

I'm thinking about making my own simple game engine and a level editor for my next game and I was wondering, how does a "build" button works for example in Unity game engine window? How are these used assets, scripts, models, textures packed? How are these being playable? Can someone explain what happens behind? P.S. I'm not sure if techniques are the same or different on each platform, but I'm looking for techniques for an android platform. And I'm not concerned to Unity only, would love to know different techniques.

Thanks.

This is a good question, I would like to know as well. To my knowledge, editor files like .unity are XML based and, upon being built, are compiled into runtime code compatible with the engine. As far as asset packages, I think they are zipped in a certain way and unzipped at runtime and the relevant assets loaded. Just guesses from what I've read in passing.

;

When you hit "build" it probably converts your assets into file formats that are optimized to be read very fast (assuming the editor doesn't do it as soon as you import it into the project), then it takes all those assets and packs them all into an archive for the purpose of blazing-fast access, compression and weak obfuscation. Each big developer has its own archive file format: id Software has .PAK archives, Square-Enix has .LGP archives, Valve has .VPK archives, Blizzard has .MPQ archives (IMHO the best ones featurewise), etc, etc, etc.

In the case of Unity it probably also compiles all your C# code with maxed performance flags and pre-parse your JavaScript as well.

Okay, just as I thought, everything is being compiled and packed into one single archive. And these executables as Toothpix mentioned are pre-compiled and just copied to a build path, right? How these executables access those assets and scripts if they are packed into single archive?

By the way, is there any links/tutorials/code examples regarding to changing assets "from easily readable and portable formats to several large binary formats usable only by the game engine"?

;

I somewhy feel your post was a little bit offensive. If I'm asking something doesn't mean that I'm total noob and don't know anything at all. Will think of a solution myself, nevermind, thanks anyway.

How these executables access those assets and scripts if they are packed into single archive?

Hi zgintasz. Will you be making a resource packer using c/c++ or a different language? I wrote a step-by-step article on a virtual file system many years ago. It more describes my (old) implementation in detail than a general literature on virtual file systems but it handles packing of resources, unpacking, and individual file access with compression.

http://mrcodesushi.wordpress.com/?s=virtual+file+system

The provided source code in those articles are now gone since i moved to a new repo. The new Virtual File System can be located here:

https://pulsetec.codeplex.com/SourceControl/latest#Source/Pulse-Tec/Include/FileSystem.h

https://pulsetec.codeplex.com/SourceControl/latest#Source/Pulse-Tec/Source/FileSystem.cpp

https://pulsetec.codeplex.com/SourceControl/latest#Source/Pulse-Tec/Include/ResourceFile.h

https://pulsetec.codeplex.com/SourceControl/latest#Source/Pulse-Tec/Source/ResourceFile.cpp

I did a very terrible job writing this article as it was just more of a developer journal for me to write stuff out in my mind but if this helps you even in littlest way, i'm glad that it did.

Awesome, thanks a lot! I'm sure it'll definitely help me.

How these executables access those assets and scripts if they are packed into single archive?

It's quite simple, actually.

Archive formats store some sort of look-up table somewhere in the file. In a very basic archive format, each entry in your table will contain the name of the file, the offset of that file in the archive (ie: where to fseek to), the compressed size of the file, and the size of the file when decompressed. It could also store flags such as whether the file is compressed, which can be useful if you're storing files that are already compressed (such as PNG images) because there's no point in compressing a file twice.

Typically, how it works is that your code will tell your I/O module to include an archive in the search path. Then, the first thing your I/O module will do (well, after maybe file-mapping the archive) when you give it that archive is to read that table and put it in somewhere in memory, probably in a dictionary or a hash table, so it knows where to find all the files. Later, when your program requests a file from that archive, your I/O code can quickly determine whether that file is contained in the archive, and it already knows where to find it. (In what archive and at what offset.)

For a very simple real-life example, here's the archive format of good old FFVII's LGP files: http://wiki.qhimm.com/view/FF7/LGP_format. It's obviously a very simple archive format. A lot of companies want additional features such as encryption or formats that make it easy to update, remove or insert files without rebuilding the whole archive.

Also, sorry if I'm stating the obvious, but Archive formats, including generalist ones like ZIP, don't compress all files together. Rather, each file is compressed (and optionally encrypted) separately, AND THEN concatenated into one single (or several) archive. Otherwise you would have to decompress the entire file to reach a file stored at the end of the archive, which is obviously not optimal.

Shameless plug but since this is related to the topic, I'm actively working on an archive format partly inspired by Blizzard's MPQ format which I am a fan of, with an easy to use C++ 11 API library to access those archives. It has/is going to have features like built-in localization support, platform-based include/exclude of files, first-class support for modding and patching, high performance file reading, multi-threaded access, etc. I wonder if there would be any interest in that kind of library because I wouldn't mind releasing it to the public if it's something that could help developers.