I''m curious, how do most of you retain compatibility with files when you change the spec? For example, say your level designers have completed half the levels when suddenly you realize that you need to change the file format to accomodate some new feature or requirement. Obviously you''d need to retain those levels (or face a mob of angry mappers! ) while still changing over to the new format. Currently I do this: when I''m first designing the level file, in the header i''ll put a couple of variables that describe the *size* of certain constructs in a file. For example, the header might look something like this: struct Foo_S { int ThisIs; int Just; int AnExampleStructureWhichOne; int MightStoreInTheFile; }; struct Bar_S { int SameWithThis; }; struct Header_S { int NOfFoos; int NOfBars; int SizeofFoo; int SizeofBar; }; The program looks at the Sizeof___ variables when they''re loading stuff in to know how big the structures in the files are -- *but* the program only uses the stuff it *needs*. If later I change the file format so that a structure of type Foo has more variables, I just change the SizeofFoo paramater in the code while still retaining compatibility. Advantages: - Backwards compatibility. An old version of the code can read a map designed for a newer version (Although it will ignore some things). - Simple to implement Disadvantages: - You can''t change the order which variables are in (without breaking compatibility). So everything you add has to be at the *bottom* of the structure. An alternative to this method is just to pack your files. This can be somewhat wasteful though. Here''s what it would look like: //structure to be stored in a file struct ExampleStructure_S { int Var1; int Var2; int Var3; //leave 16 bytes of padding which can be used later char Padding[16]; }; Also, I imagine you could also store it as a text file. Although this could be slow and big, it is probably the most versatile of all these. Finally, you could just write a converter utility. This is easy (although if the spec changes a lot, somewhat annoying). My question to all of you: which of these methods do you prefer, or do you handle this some other way? --TheGoop

I thought of this once and I just said "Screw it!" My idea was to just include header for each struct. I''ll try to do it in c/c++

struct somethingheader
{
int versionnum;
}

struct something
{
int crap;
}

struct something_ver2
{
int crap;
}

void readstuff()
{

// Variables
something SomethingVar;
somethingheader SomethingHeaderVar;

// Now get the file with our readclass
readclass->doreadthingy(SomethingHeaderVar);

if (SomethingHeaderVar.versionnum == 1)
{
// We can read it because it is version 1
readclass->doreadthingy(SomethingHeaderVar);

// else: ignore
}

}

Excuse my terrible coding. I''m a VB programmer. Anyhoo, it basically reads the info only if it is an old structure. That''s your backward compatibility. Hopefully the new struct is not an important struct. Another solution would be to tag the whole struct. If you are going to do that, you might as well just read a text file. A hard-to-manage solution would to use seperate variables. Check this out:

struct VariableStruct
{
int VariableType;
chr doeswhat; //This is used to tell the
//program what variable it is used for
}

struct IntStruct
{
int crap;
}

struct ChrStruct
{
chr crap;
}
...
...
...
You get the point. Just read the header first and then check what''s the datatype to retrieve and the type of datatype. Like brushcoord, entity, whatever. I''ve never tried this one, but I don''t need to because I''m not including compatibility.

My answer is to never save anything as binary that may get changed. (As a side-effect, this gives me portability, the ability to view my data in a text editor, etc.) I save and load as text, usually in keyword/data pairs to let the load function know what it is reading. The basic algorithm for my simplest files is like this:

while (file >> keyword){    if (keyword == "FIELD1")        theObject.variable1 = ReadString(file);    if (keyword == "FIELD2")        theObject.variable2 = ReadInt(file);    // etc, for all fields} 

I clean this up a little by using a macro to eliminate all the repeated if checks. This works great for version compatibility: delete a field, and it will (nearly always) be successfully ignored. Add a field, and it can appear at any point in the file so you can add it in at the end. Even more effective is when you can specify a default value for new fields, so that simply loading the database in and saving it out again will update your files.

However, I am sure there are ways I could improve on the above method, I just haven''t given it much thought yet. Anyone got any good ideas on being able to read files in a versatile manner, especially where the order is not fixed, fields can be of different types, and fields are not mandatory?

In fact, to slightly 'clean up' the code for the method I just showed, it could be done this way:

while (file >> keyword){    if (keyword == "FIELD1")        Read(file, theObject.variable1);    if (keyword == "FIELD2")        Read(theObject.variable2);    // etc, for all fields}// And overload Read() for all the types you intend to save// Read loads in the appropriate type and stores it in the//   reference variable you passed invoid Read(istream, int& theInt);void Read(istream, string& theString); 

This will simplify writing new loading code, yet still produce a compiler error if you try to load an unsupported type, which is how you want it. Technically you'd be violating encapsulation by passing a reference to a member variable, but since you could only ever do that from a member anyway, this isn't such a big deal.

Edited by - Kylotan on May 25, 2000 7:43:28 AM

Kylotan, that''s pretty much what I was talking about but you did it better.

When in development, save all your data in easy-to-read text files. That way when the spec changes, you just change the parser''s rules. 5 seconds and you''re done.

When you deploy, then move it all to compressed binary files, etc.

You can make the file format even more editable by using some standard format such as XML - which I use for all my projects. Another format is SGML, but in general XML is better for this purpose because it is simpler.

XML is a HTML like format that allows you define your own tags. The great thing is that free libraries exist that can parse and write these files based on simple rules and convert them into data. This makes it easy to write converters and make things backward compatible while avoiding irritating parsing details. The advantage is also that you can use XML editors (such as XML Spy) that can make sure that your files (while you write them) observe the grammar.

The only drawback is that XML is larger than binary files, but when the game is done you can convert it all to binary formats to gain the extra speed.

XML may seem to be an awful overkill to learn, but in reality it is very simple if you some html.

Another advantage is that designers can begin to make content even before any map editors have been made. They can simply use the XML editor. A real map editor is still needed though to improve the content creation process.

Jacob Marner

You can make the file format even more editable by using some standard format such as XML - which I use for all my projects. Another format is SGML, but in general XML is better for this purpose because it is simpler.

XML is a HTML like format that allows you define your own tags. The great thing is that free libraries exist that can parse and write these files based on simple rules and convert them into data. This makes it easy to write converters and make things backward compatible while avoiding irritating parsing details. The advantage is also that you can use XML editors (such as XML Spy) that can make sure that your files (while you write them) observe the grammar.

The only drawback is that XML is larger than binary files, but when the game is done you can convert it all to binary formats to gain the extra speed.

XML may seem to be an awful overkill to learn, but in reality it is very simple if you some html.

Another advantage is that designers can begin to make content even before any map editors have been made. They can simply use the XML editor. A real map editor is still needed though to improve the content creation process.

Jacob Marner