So I did a bit of searching around through the forums here and on google and I can't seem to find much discussion about how to organize an MMORPG on the server side. I'm not talking about how the database stores the information, but rather how it is structured in memory (lists, trees, ?). I've been sort of playing the armchair mmorpg designer over the last few months thinking of how the different systems would be organized and such and here is what I was thinking... For arguments sake lets just pretend that this mmorpg runs on a single server to keep things simple(r). I broke the threads up into the following: Logic - processes user/npc commands, updates coordinates, checks for collisions, etc Maintenance - cyles through all world entities and releases deleted items as well as doing backups of dynamic world data to a database NetworkClient - acts as a bridge between a player that is connected to the server and the physical object in memory that represents them, processes/validates incoming player requests as well as sending out world state updates NetworkListener - listens for new player connections, authenticates users, adds players to the world and spawns a NetworkClient thread to service them NPC - does the "thinking" for the non player characters, processes environment stimuli, generates responses, etc Thats a very brief overview of how the flow of execution would work, but here is what I came up with for how to organize the world: World -2d array of sectors (each sector being a square chunk of the world) -Global list of ALL world objects (used by the maintenance thread to cleanup dead items and save changed ones to the db) -Global list of NPCs (used by the NPC thread to cycle through each one and "think") Sector -Local list of Items/Objects in this sector -Local list of NPCs in this sector -Local list of PCs in this sector A sector would ideally be about as wide/high as the players visual distance so that you could have a buffer of the 8 sectors surrounding the players current sector. I stole this idea from how Ultima Online handled this in their tile based scheme. The benefit of the overlap in lists, is that you can both cycle through all objects/npcs when you need to or you can limit yourself to a smaller partition with the local sectors. For example if an event happens in sector(x, y) you would want to add the event to all the PCs/NPCs that were close enough to hear or see it, so you would only cycle through that sectors pcs/npcs and maybe the 8 surrounding sectors. With the localized lists you basically have a much smaller search space. One of the problems I can see is the excessive amount of locks that would need to be performed so that the threads don't corrupt the data (is this inevitable in a large system like this?). So anyway, thats what I've been tossing around in the old noggin the last few months or so. I'm curious what ya'll think about anything related to this? Questions? Comments? Criticism (constructive )? [edited by - idrifter on December 1, 2003 10:54:11 PM]

Many problems I see:

- 2D array of sectors? This could be a quadtree

- What about the world geometry, is it dynamic, do players have to download the entire map?

- You do not have to differentiate world objects and "NPCs". All
interactive objects can be entities.

- Why a "maintainance" thread? You can just eliminate entities.

- How do you handle the players, the ones that are online, your database?

- At one thread per client, you will kill your system

Those are only a few. I believe you should start by designing a multiplayer game and implementing it fully. It seems like you are not thinking about many of the important issues. You don''t need threads for everything. Usually, you want to have a tree which has a geometric based distributions of entities and world geometry and some global lists of all objects and all world geometry.

---

Looking for a serious game project?
www.xgameproject.com

The sector is a good idea. You should have some sort of cache system for these sectors though.

There's also the concept of an invisible "ring" (it can be a square to simplify things). This ring, which every client has (stored server side), decides what a client will recieve. If something is happening outside the ring, the client wont see it, hear it, or know that its even happening (unless a chat system exists, and another player outside of the ring informs the client of the event happening).

Everquest did this, except these "rings" were static, and called zones. It's quite possible to program the rings to move along with the client. As the client moves, his ring stays with him, and things can enter/exit the ring (enter/exit memory). This ring would also be used for determining how far the client could see and hear. It doesnt have to be that way, but its convienant, depending of course, on the size of the ring. The bigger the ring, the more memory/more bandwidth required. The best method is having two or three rings, each differing in size, each having its own purpose.

You HAVE to use a method like this to build an MMORPG. It is impossible to send EVERYTHING that is happening in your world to a client (and this should be obvious). For you people that believe nothing is impossible, yes, its possible to send everything thats happening in your world if your world, by definition, is as small as the ring.

As far as sending world geometry, it can be achieved. Neverwinter Nights achieves this quite elegantly, when you enter a custom module, the entire map is sent to you in about 2 seconds (its about 1 kb). That is because its tile based, and it only has to send an area definition call to the client, which contains id numbers for each tile. This is quite possible, if you use tiles. If you intend to send vertex data through a network, depend on having load times of up to 10 minutes or longer on DSL lines. If you do something like most mmorpgs, the world "data" (as in graphical data) usually exists on the client side, contained in some obscure file somewhere deep within the game's folder structure within the operating system. The file is usually designed so that it wont be easily tampered with. Most mmorpgs have a checksum im place to ensure that the file is never changed (check date/size/etc), and if the check fails, the player is denied access from the game. Its up to you to tell the player why >.

I hear that a lot of mmorpg's depend on databases for almost everything thats going on in the world. I've also heard that other mmorpg's only save world state to database (in some save thread) every so often. Im not sure which is the best method at the moment, as I havent even gotten to the part of programming the entire network system. Im still on a 3d engine (which is completely client side). I'll cross that bridge when i get to it.

i will now stop rambling.

[edited by - fireking on December 1, 2003 12:03:49 AM]

quote:

---

Original post by Max_Payne
Many problems I see:

- 2D array of sectors? This could be a quadtree

- What about the world geometry, is it dynamic, do players have to download the entire map?

---

I haven''t put much thought into how the world geometry would be organized, but obviously that would be another big consideration.

quote:

---

- You do not have to differentiate world objects and "NPCs". All
interactive objects can be entities.

---

The main reason i singled out NPCs was because presumably you would want to cycle through them to update their state, so why not have a specialized list instead of searching through everything?

quote:

---

- Why a "maintainance" thread? You can just eliminate entities.

---

Deleted items might need to be removed from the database, so thats why i created a special thread for that type of stuff rather than blocking on db queries in other threads.

quote:

---

- How do you handle the players, the ones that are online, your database?

- At one thread per client, you will kill your system

---

The way I had it setup, a thread was spawned for each player, but you could always have a thread manage multiple players.

quote:

---

Those are only a few. I believe you should start by designing a multiplayer game and implementing it fully. It seems like you are not thinking about many of the important issues. You don''t need threads for everything. Usually, you want to have a tree which has a geometric based distributions of entities and world geometry and some global lists of all objects and all world geometry.

---

Thinking about this stuff is more fun and educational than making another multi-player tetris clone even if I never finish anything

quote:

---

Original post by fireking
The sector is a good idea. You should have some sort of cache system for these sectors though.

There''s also the concept of an invisible "ring" (it can be a square to simplify things). This ring, which every client has (stored server side), decides what a client will recieve. If something is happening outside the ring, the client wont see it, hear it, or know that its even happening (unless a chat system exists, and another player outside of the ring informs the client of the event happening).

Everquest did this, except these "rings" were static, and called zones. It''s quite possible to program the rings to move along with the client. As the client moves, his ring stays with him, and things can enter/exit the ring (enter/exit memory). This ring would also be used for determining how far the client could see and hear. It doesnt have to be that way, but its convienant, depending of course, on the size of the ring. The bigger the ring, the more memory/more bandwidth required. The best method is having two or three rings, each differing in size, each having its own purpose.

You HAVE to use a method like this to build an MMORPG. It is impossible to send EVERYTHING that is happening in your world to a client (and this should be obvious). For you people that believe nothing is impossible, yes, its possible to send everything thats happening in your world if your world, by definition, is as small as the ring.

As far as sending world geometry, it can be achieved. Neverwinter Nights achieves this quite elegantly, when you enter a custom module, the entire map is sent to you in about 2 seconds (its about 1 kb). That is because its tile based, and it only has to send an area definition call to the client, which contains id numbers for each tile. This is quite possible, if you use tiles. If you intend to send vertex data through a network, depend on having load times of up to 10 minutes or longer on DSL lines. If you do something like most mmorpgs, the world "data" (as in graphical data) usually exists on the client side, contained in some obscure file somewhere deep within the game''s folder structure within the operating system. The file is usually designed so that it wont be easily tampered with. Most mmorpgs have a checksum im place to ensure that the file is never changed (check date/size/etc), and if the check fails, the player is denied access from the game. Its up to you to tell the player why >.

I hear that a lot of mmorpg''s depend on databases for almost everything thats going on in the world. I''ve also heard that other mmorpg''s only save world state to database (in some save thread) every so often. Im not sure which is the best method at the moment, as I havent even gotten to the part of programming the entire network system. Im still on a 3d engine (which is completely client side). I''ll cross that bridge when i get to it.

i will now stop rambling.

[edited by - fireking on December 1, 2003 12:03:49 AM]

---

Yes, that idea of a "ring" or radius got me thinking of how to organize or sort players by their location to each other. The sector idea is rather primitive but from it you could create a subset of who is impacted by what and refine it further with actual distance calculations between the two entities or something similar.

Also have a look at www.darkandlight.com. They have a world database that contains data in which the world can be drawn at any location, with any level of detail. Im not sure if its actually chucked across you''re internet connection line, but that would be quite nice if thats how it is. Imagine the possibilities! Players could get used to hanging out by a tree somewhere within the world. Some day, a fight between a dragon and an ancient wyvern could break out, and knock the tree over. The players would come back to find the tree on the ground, only to go looking for a new place to hang out.

I programmed an MMORPG, and it was pretty successful (last user count was 800 in database). The thing that the players complained about the most was the fact that they were getting bored. And thats because the world was static. You could go to the same old dungeon, and see that same old skeleton dood in that same old corner. Change is enevitable, and if your world doesnt change, things dont feel right, and players get bored. No matter how big you make it, some day they''ll notice that things are always the same everytime they login.

Change is also bad. Look at ultima online, it changed so much that I dont even like playing it anymore, its like a completely different game. The changes need to be balanced and justified (or realistic).

Programming a system that will allow this to happen effeciently is the challenge that every programmer in this field is constantly battling...

quote:

---

The main reason i singled out NPCs was because presumably you would want to cycle through them to update their state, so why not have a specialized list instead of searching through everything?

---

Actually, in an entity based system, all entities can potentially be updated, there is no reason to deny your world objects to have a behavior which relies on an update. NPCs can be entities, just like elevators, doors, items, lights, etc.

quote:

---

Deleted items might need to be removed from the database, so thats why i created a special thread for that type of stuff rather than blocking on db queries in other threads.

---

Actually, if your database calls are blocking, a DB query thread would probably be a better idea. Everytime you have a DB query to do, you queue it, and that thread takes care of it. The fact is, you will need database calls for other things than removing objects.

quote:

---

The way I had it setup, a thread was spawned for each player, but you could always have a thread manage multiple players.

---

You could actually perform all the player updates at once, in the same thread.

quote:

---

Thinking about this stuff is more fun and educational than making another multi-player tetris clone even if I never finish anything

---

Who talked about a tetris clone. The fact is that you need experience. I don't want to see you post another MMORPG post in the help wanted forum and get flamed because of design flaws, but still manage to recruit people and realise its a too large undertaking two months later. It seriously stinks to see dozens of people try this and never come back, they just make a bad reputation to the indie game development community.

To make an MMORPG, you need to make your own MMORPG engine, and for this, you need game engine design skills, this can only be acquired with experience. Of course, some might suggest you to start with tetris, I say, why not try making a simple FPS games, with alot of restrictions and try something around that, its simpler than it sounds. You could also try making a 3D tile engine (they are simple to make) and you could try making this multiplayer.

I say, try at least once, with your first 3D engine, you will be very disappointed in many aspects, but you will realize that some of the things you originally wanted just don't work in reality, it will allow you to perfect your design and come with new ideas that actually well in the real world.

You know what? I'm starting on the design of my second 3D engine, after 3 years of work on another engine that will not actually be used in a game. It was some very good learning. This engine can only be better than my previous one. I think I am fairly skilled and MMORPGs attract me, just like every dreamer-programmer out there, but I will not attempt making one until I am sure I can achieve it.

---

Looking for a serious game project?
www.xgameproject.com

[edited by - Max_Payne on December 2, 2003 2:02:21 PM]

WAY too many threads. You''ll never get that to work. You will get millions of "random and unreproducable" errors. And even if you did manage to get it to work, all the state changing would kill your performance.

Multithreading should be used sparingly, if at all. I''ve rarely seen a good reason to multithread. the CP detection part of IOCP is one good reason. AI is another (AI on the level of a chess player, not a MMORPG creature who is only monitoring their immediate environment for threats, heh).

Your design seems fine to me in most respects, but you should retool it to be single threaded. You are unnecessarily complicating what is already a very complex program, while at the same time killing your server response time.

In my opinion, anyway

Currently in my setup, for the zone server (the type of server the client is connected to 95% of the time) I have 3 threads.

The first thread is the main thread and basically all it does is handle messages from the world server.

The second thread just basically listens for new client connections. Once it gets data from a new client it creates a new client object and adds it to a list of entities, then follows the next behavior. If its data from an existing client it parses the packet and pushes it on to the clients InQueue.

The third thread basically iterates through the list of entities (Clients, Mobs (monsters), NPC's, and misc entities such as doors and switches. As it hits each entity in the list it calls its Process(); method. For clients the Process method pops messages of the InQueue and handles them appropriately. If any new packets need to get set in response, theyre created and pushed on the the OutQueue for the client. After all the new messages are handled, the Process(); method finishes up by checking a few timers (to see if it needs to resend packets or if the client has timed out due to lag and whatnot) and flushing the OutQueue by sending all the outqoing packets.


I havent fully tested this with a lot of "real" clients, but so far so good. Honestly, if i get 30 people connected with no problems I'll be fairly happy. It's my first go at it and I don't expect it to be perfect and I already have a few ideas on how to do it better/differently.


-=[ Megahertz ]=-

[edited by - Megahertz on December 3, 2003 11:52:13 AM]

[edited by - Megahertz on December 3, 2003 11:53:09 AM]

quote:

---

Original post by Megahertz
Currently in my setup, for the zone server (the type of server the client is connected to 95% of the time) I have 3 threads.

---

I also use three threads in my server. Two are running the exact same code though These two do nothing except handle socket completion notifications and dump the completed buffer into either the "available for use" pile (if it was a completed write) or the "to be handled" pile (if it was a completed read). That''s all they do, it''s a very small bit of code actually, but vitally important

The other thread is the "everything else" thread. Essentially its job is to deal with the "to be handled" buffer piles, sending out responses to the clients as required, and to take care of a bit of housekeeping (like saving characters to the database every so often). Eventually it will also deal with MOB and NPC AI and a few other things.