To respond to KalvinB''s question...

Artifact uses a method like I described. 1 file per player in a special "player" directory. We limit player names to alpha characters and spaces so that their account name is guaranteed to be a valid file name. On top of that, the information is stored in a text format, with one "data element" per line of text (that''s what happens when the server programmer''s background is traditional text MUDs ;-) ).

Game information is saved once per minute (or thereabouts, depending on various settings). Player information is saved less often, but I don''t recall the specifics, and it''s not that much less often. The entire player database is backed up twice daily. The player database backup is a separate process from the lobby and game servers that just grabs all the files as they currently are and tosses ''em into a CPIO archive.

As for concerns about the game crashing...we generally go for weeks without a game crash. And there are *months* between serious system crashes that cost us player files, and those have usually been OS problems (like the Linux file system falling apart spontaneously) rather than any fault of our own.

Artifact isn''t anywhere near the size of a game like EverQuest, but we peak from 120-150 users in prime time hours, averaging from 75-100 players 24x7. There are generally 800-1200 players that login and play in a 24-hour period. And we estimate our maximums are about 3-5 times what we currently see. And if we exceed that, we can always upgrade our server hardware to accomodate more. RAM is cheap, hard drive space is cheap, and processing power keeps getting cheaper.


DavidRM
Samu Games

Kalldrex, write a program that loads a file with ifstream then pull the plug on your computer to simulate a crash. Power it back on and see if the file is still there.

DavidRM,

That''s pretty much how I''m doing it. The autobackup takes about 13seconds to complete so I plan to only have it done every hour.

I''m not expecting Tombstone to reach EverQuest proportions but it''s written to be expandable to handle as many as sign up.

Ben
http://therabbithole.redback.inficad.com
http://www.vendettaonline.net

The debate seems mainly to be between: SQL vs. Files.
How much faster is files compared to SQL if they
amount to 10000, or are you splitting them up in
multiple directories like a dictionary? A,b,c,d,e,f,g...

Also are you reading the entire file from top to bottom or
are you random acessing and retreiving just the line you want.

I think i''m going to use files since MySQL++ doesn''t like MSVC

Wow, this turned into a holy war of sorts. Anyhow, I''m still in favor of using a SQL database, for a couple of reasons. You can do online backups of the database, benefit from whatever caching setup your chosen database supports, transactions, rollback, it''s everything that you will eventually find out that you need for an MMORPG. A directory / file structure isn''t a bad idea, it''s just going to require more work to maintain it than with a database server.

And just to explain, this doesn''t mean accessing the database every time you need to use a value. When a World loads, it loads whatever persistant information it needs from the database, and manipulates it in memory until the world server is brought down. Periodicly saving any data that might change. When a user logs on, their information is loaded from the database and stored in memory until they log off, again, periodic saving.

So with this setup, your database method (file or SQL server) is not being accessed during time critical operations, only for world loading and user loading (and various other times depending on game design). So performance, while important is NOT critical, I''d say that picking the option that will give you the features you need is much more important than mind-numbing speed (which I''m still not sure would be significant in using files over a SQL server).

I''m a proponent of SQL but I grant that it can be more applicable for some types of games than others. IMO, for a persistent-world game, SQL is definately the way to go. Someone earlier had said as a rhetorical question (with the answer assumed "no"): "are you really going to run queries across player data?" For a persistent-world game, my response would be "absolutely!" Look at the problems current MMORPGs have had with exploits and bugs. Asheron''s Call has had numerous problems (too-powerful "broken" items after an update, etc.) where they have been unable to make retroactive changes because of the nature of their filesystem. With SQL and a properly designed database structure, it would be completely possible to modify the game world and to do so without even taking the world down. And again, scalability is never a problem. While a flat-file structure is fine for smaller games, they don''t scale up well. SQL incurs a higher initial and recurring cost, but allows for more power, flexibility, and scalability than a flat-file system could provide (barring a comprehensive custom database system written specifically for the game, which would equate to considerable additional development cost).

For large (multi-server) worlds, programming a flat-file solution is going to have to include a number of features that SQL provides for free, the most significant being transaction management. Since a single dataset must be shared across multiple machines, some system must be in place to serialize access to files to avoid corruption. Once you begin to consider issues such as resource starvation (the dining philosophers problem), it seems IMO to make a lot of sense to just move to a database.

Say you give an item to another player... the transaction takes place in memory and is queued to be sent to the database. The database transaction confirms the transfer. No duping is possible regardless of whether the server crashes or not... worst-case is just that the transaction is rolled-back and effectively never happened. Again, this is certianly possible in a flat file structure, but transactions with rollbacks (in the event of a server crash) IMO are not... at least without a separate transaction server process, which is effectively a database server. Thus duping becomes an issue as well.

As was said earlier, both systems are meant to be used in the same way. That is, the game server would maintain the bulk of applicable data in memory rather than querying the server every time it was needed, but regular synchronization would still occur. Remember that both a flat-file or SQL system are meant to be the backup and authority on what is what -- they address the problem in the same way. If you try to query off your SQL server every time you need to know something, expect the game to run abysmally slowly.

Switching over to SQL if needed wouldn''t be much of task as most game information is already in Excell format. The rest could be changed over pretty easily. But until more than 2000 people are consistantly playing at once I''m not going to worry about it.

Ben
http://therabbithole.redback.inficad.com

It''s a fact that people will defend their choice of tool...to the death, if necessary.

Assumptions:
1. The game state, either for the entire game or for a particular "zone", will be in RAM. This assumes a real-time game or a game where real-time resolution is necessary.
2. A copy of the game state, as above, will be kept in some form of secondary storage.
3. For performance reasons, #2 will generally lag behind #1, but will be updated at periodic intervals, either in part (delta) or in whole (full).

If you reach a point where the game state has to be rolled back...you''re screwed already. This is true whether your secondary storage is in a fully-relation database or scribbled on index cards in a file cabinet behind the receptionist''s desk.

With few exceptions, only things you''ve already thought of will be caught and prevented by built-in checks-and-balances. "Exploits" are, by definition, achieved via a loophole in the system, so it''s likely that a lot of "transactions" and other changes to the game state will occur before the use of the exploit is detected. Is a "rollback" even an option? Probably not. More likely, the necessary correction in the game state will require an intervention. An intervention means that the state of the game will have to be "forced" back into shape via updates to player data, game object data, and so on, in their current state.

Would a database help in this situation? Possibly. But what will really make the difference is the administrative tools you''ve built into the game for use by designated "admin" (or "GM") accounts. Tools that include both research and state changes.

In fact, its the tools built specifically for administering the game that make the most difference in nearly every situation you want to mention. And whether the secondary storage is a database or not doesn''t really enter into the equation.

So should a game use a full-blown database server or just hire gnomes to quickly interpret the bitstream to Sanskrit for chiseling onto clay tablets?

My recommendation in this, as in just about any other topic, is: Use the tools you are familiar with that allow you to achieve the desired result. Use what works for you. Adapt as necessary.

And keep an open mind.


DavidRM
Samu Games

Well so basically when the server is activated it queries the sql server which basicaly puts it in RAM correct?

Then if the game server crashes it won''t loose the database just anything before the past save! Maybe do a save every 15 minutes or something and hopefully fast enough that noone notices. I think now that i''m not using MSVC i''ll use SQL cause it''s easy to administrate and to check! I might keep a database of logs too so we can check things out but that''s a lot of quering so ???

First I''d like to say that DavidRM''s post was excellent. I support everything he said wholeheartedly.

From a practical standpoint, you''ll probably end up doing something like this:
* when the server starts, read game state info into RAM, etc.
* when a player logs in, read their information from the file/database into RAM
* when a player does something noteworthy, stick the transaction in a queue

There are a ton of different strategies but the one I''d likely use would be a unified transaction queue per process. Then you have a thread that just processes the queue while there are transactions available. You keep concurrency and locking issues down and ensure that everything is processed in a timely manner. If the server ever crashes, the only changes you lose are the ones still sitting in the queue. With this method, "saves" are never neccessary because you''re perpetually updating the game state with pertinent changes. Transactions become handy when you do things like have a player giving an item to another player. If the server crashes halfway through a tran, it''s automatically rolled back so no data corruption occurs.

One reccommendation: if you''re going to use SQL, abstract all your operations into stored procedures. They''re pre-compiled and run much faster than standard queries, and they abstract you from the underlying data design. If you ever want to query a table directly, use a view instead, for the same reasons.