Thanks in advance for reading guys. And thanks again to those who emailed me about network middleware. You were very helpful. I just want to get some feedback from someone who's been doing some datastream work or even concepts so they can correct me if I am heading for a carwreck. Basically my game client: 1)conects to the an authentication server(if authen passes then continue. this server also handles account editing and creation.) 2) Then to a patch server(if patch req then patch then restart if necessary. Else continue.) 3) Then to the player DB for player selection / creation etc. (Upon player selection continue) 4) The "zone server" begins streaming data to the client as to current zone entities and their states. Anytime any entity in the players field of influence changes the client is updated. Here is the real question: For security reasons to slow down hacking and cheating all the client does is send "state changes" to the server. A direction key is pressed and the server is sent a data package with the clients change in heading / speed etc. When the client equips an item all the server is sent is a packet stream stating that the client has requested item from inventory position xx be moved to equipment position xxx. My theory here is that no information is sent directly from the client to the DB. The client can't say for example that he is equiping "A holy sword of asskicking" all his client sends to the server is a request to check his player DB and equip the item at pos x. The same with stats. When a spell is cast on a player to buff them, the client doesnt send a request to increase his dex 50 points the client simply sends a request to the server saying I have cast spell #623 in my spellbook on "target". The server then checks the players spellbook database, confirms the player meets the requirments to cast the spell based on the player DB and adjusts the stats in the players DB for the effects of the spell. Ok there really is a question here sorry Just needed to properly describe the datastream. The obvious traffic load is going to be directly between the client and the zone server and the zone server will co-ordinate internally with the various DataBase servers. Obviously there is going to be a huge crunch on the network backbone on the zone servers. Does anyone think it is going to be bad enough to the point that I should look at allowing the client to request state information directly from the DB servers? To be honest Im 99.99% against that as it just screams out for people to harrass your DBs. My current shard design is that the authentication and patch server applications will actually run on the same physical server and will be directly accessible on the Internet via a public IP firewalled to the 4 ports required for the authentication and patching process. The Zone servers will also be directly available to the Internet on various IPs firewalled to teh 2 ports required for communication to the clients. Currently my design has a gigabit ethernet blade for every zone server exposed directly to the bandwidth backbone on the ISP and a second gigabit blade IP'd for the internal LAN mask only. All of the DataBase servers will run single Gig EtherBlades that plug them into the internal LAN cluster. I am trying to create a stable alpha environment using this design starting at approx 30 players. My alpha goal is to optimize the datastream for up to 100 players per zone. Can anyone tell me if I am completely overlooking some fundamental detail that is going to break me. This next step I take is a doozy and I'm going to blow another thousand dollars on switching hardware once we take the next step so I sure hope I'm not barking up the wrong tree. =) Thanks in advance guys. Mark PS: Aside from the various DB servers I do also have combat, AI and Spawn control servers that will reside on the private LAN mask but I'm really not as worried about the internal traffic structure so much as I am about the datastream between client and zone/server. I have concidered stream combat data directly from the combat server to the client as far as combat rounds and results but all combat state changes will still be initiated through the Zone server. Aakrana: The Forgotten Lands Currently seeking help from: C++ coder ZeroC / ICE programmer [edited by - Slyxsith on January 14, 2004 10:34:46 PM]

It depends on many things, as to wether your heading for a car wreck.

It sounds okay, if the servers are up to it.
Definatly more secure, unless someone breaks your protocal.
Most systems farm out some of the calcs out to the clients.

Your "Zone" servers are going to have to be _VERY_ chunky.
Mainly in your memory.

Which data base system are you thinking of? (Mysql, Oracal or custom)..
How much data are we talking in the DBs?
Will there be files sent or will it all be records in the dbs?

The main thing would be, whats your fall over plan.
Server goes down or too many playes in one "Zone" type problem recovery?

My Two cents worth.
More questions than answers but ehh.

But I''ve worked on, and currently working with servers to handle many clients and data transfer.



Armand
-------------------------
It is a good day to code.

Hey Armand,

Thanks for the quest.. err I mean reply

Starting alpha I''m targetting approx 30 clients with a 3kBps (got it right this time) datastream. My pre-beta goal is to optimize up to 100 clients per zone server.

My database for alpha is infact mySQL.

The databases for alpha are going to be pretty big.
The PC database will have 1 record per character and approx 600 fields per record. (The actual table structure is not finished yet.) 6 players per account and 500 accounts for alpha. Average field size is 14 bytes. 6x600x500x14 = *grabs a calculator* 25 megs.. (someone correct me plz)

The item database in alpha is capped at 10k with 60 fields per item and an average field size of 8bytes. 10000x60x8=~5MB

Other significant DBs will be the M(onster)o(r)B(east) DB which is currently 10MB, the Loot table DB currently 3MB but I anticipate this to quadruple once we start testing the spawn engine. Also the Quest DB which again is tiny now but will probably end up being the largest of the DBs at 100ish MB by end of alpha. The quest DB is not a huge concern as it''s access will only represent 5% or less of the datastream communication.

Increase those sizes by approx 25% for DB overhead.

Based on my current projections, MoB, Player, Loot and Item DB access are going to be massive. Based on a moderate zone with 200 mob entities and 50 attached clients I anticipate the Zone server to average 50ish LAN DB quieries per second. That allows for players checking stats, inventory, looting corpses, adjusting equip etc. However based on the nature of MMRPGs the QPS (quieries per sec) will spike to well over 500/sec or more.

Based on my network knowledge I think that the true Gig LAN can handle the DB enquiries but on top of that we have double again that ammount of traffic from the chat server, combat server, AI server etc.

So I guess my real concern is hitting the Zone server with approx 100 MB /sec of datastream =) I''ve been a network engineer for 14 years and to be honest I''ve never worked on a server that had to sustain that kind of datastream. I''ve got an engineering Lan that runs gig and we spike it every once in a while but I''m wondering what anyone else thinks about the real world ability of a server to negotiate 50-100 Megs a sec.

I guess it really wont be that bad.

Bah my head hurts. Hope this makes sense. thanks for the feedback. Must eat food. =) Wife is going to throw diner at me if I dont eat soon.

Mark






Aakrana: The Forgotten Lands
Currently seeking help from:
C++ coder
ZeroC / ICE programmer

Wait a second.. how exactly do you figure a solid 100 MB/sec per zone server? 3kBps per player (and you should probably aim for a lot less than this on average, bandwidth costs money!) * 100 players per zone server is (max) 300kBps - roughly 341 times less than your 100MB/s figure. Also, those databases sizes are not at all big. Most, if not all, of the data in those DBs could be cached on each zone server. Give the servers 1GB RAM and you''d still have a lot of breathing room after caching 120MB of data. This would avoid transferring a lot of data over your backplane and eliminate waiting for the database to respond. The main issue you''d have is with data that could change while the servers are up that would need to be propagated across multiple zone servers. That shouldn''t be an issue with Mobs, loot, or quests... only player data, and totally up-to-the-second player data isn''t usually needed by zone servers other than the one that player is utilizing. With properly organized cached data, 5000 queries per second would make an insignificant impact on CPU utilization, let alone 500 or 50.

Whichever way you do it, I doubt you''d run into any problems due to network bandwidth limits (especially if you use gigabit ethernet.. and you might as well). Definitely don''t let the players access the DB servers directly. That''s a nasty hack begging to happen.

Oh, and good luck
-nohbdy

I''ll start out by saying that I''m working on a similar project (slowly). My client is nowhere near yours, but my server looks like it might be a little further along.

I think I can say with a resounding and confident "NO" that you should NOT allow the client direct access to the database (no matter what). Every entry into the database needs to be carefully verified server-side. The database server therefore needs to be hidden behind your firewall and the client shouldn''t know it from the tooth fairy.

I''m planning that the server (which you''re calling a zone server) will cache requests to and from the database server in the background (on a separate thread). For example, as a character gets closer to an unpopulated area, the server will begin pulling information out of the database into memory. As they leave the same area, that information will get marked for expiration (provided it''s still unpopulated). The server will always work from memory so aside from the caching module, it won''t even know about the database server. A side benefit to this is it puts all your database queries/operations in one place (vs. scattered throughout your code).

This is just an aside, but you should possibly rethink your database design. 600 fields in a table SCREAMS that something is very, very wrong. Are you familiar with normalization? If not, normalization is a fundamental concept in relational database design and means that each table needs to have as little information as reasonably possible (without negatively effecting performance). So for example, say initially we give each character 2 attributes, STR and INT. With an non-normalized design, our character table might look like:

name
desc
class
attStr
attInt

Now, this would work with 2 attributes ok, but it''s not very flexible. A new attribute(or 595) is going to require you to modify your table structure each time. A better, normalized way is to break that table into at least two separate tables. For example, I''m planning something like this:

-----------------
characters
-----------------
id
name
desc
class_id

-----------------
attributes
-----------------
id
attribute
description

--------------------
attribute_modifiers
--------------------
id
attribute_id
item_id
value

So now, to modify an attribute''s label or description, I don''t have to recompile code or change any tables. All I have to do is update or insert a record. If I want to set a character''s strength to 20, I just lookup their id, the attribute''s id, and insert/update the attribute_modifiers table with the value 20. As you can imagine, this can sometimes bring with it performance overhead, but if done right, it will more than make up for it in the grand scheme. For example, now this same table can be used to dynamically change attributes on weapons, scrolls, doorways, whatever (provided each item has a unique id). It also makes it possible for you to allow non-technical people to make changes to the design using a web-browser or other GUI (which admittedly is not always a good thing)

I don''t know how much database stuff you''ve done, but you''ll soon find that a table with that many fields is a NIGHTMARE to maintain and sync with code. It''s also very limiting when you start reporting out of your databases. With the first example, you would have to hard code exactly which fields were attribute related and any time you added a new attribute, all of your attribute related reports would have to be updated.

See what I mean? I probably did a poor job explaining that, so drop me a message if you have any questions or if you just want to "talk shop". When I discovered how to properly build relationships between my data, it changed my life. I think I even cried a little bit.

Your project is looking great and I applaud you guys on your work so far. I now know all TOO WELL how much effort is involved. :o

bpopp (bpopp.net)

Thanks Bpop for all that database info.

I am NOT a DB guru but I do know about proper record and table handling =) I know correct table structure inside each record will be crucial and thats why I mentioned I havent finished the table structure yet. It''s on hold untill I start getting a working network transport so I can do some benchmarks.

Currently the tables are broken into stats, abilities, training, quests, inventory and faction. My 2 biggest concerns are teh faction and inventory entires. Inventory will be manageable but faction is going to be a real cracker. I''m probably going to end up breaking down faction by zone so it''s less data to parse. Basically have the server parse the current zone factions first (should hit 80% of the faciton checks in the zone.) The remaining 20% will require the parsing of all of hte faciton tables.

Anyways thanks a mill for the feedback. I may get in touch with you when I get a chance to stop my head from spinning. My cell started ringing at 7AM this morning and I''m not a happy camper. =)

Thanks guys.

Mark

Aakrana: The Forgotten Lands
Currently seeking help from:
C++ coder
ZeroC / ICE programmer