I am just thinking of how I should be architecting a server in which it has to serve clients, and also able to receive requests coming from other servers. My network programming experience is limited, so I am having problem picturing this in my mind.

Clients connect to the server, and maintain active TCP connections. In the application code level, this is represented by an object or some handle. Other servers will talk to this server, and each will also have its own active connection objects.

Suppose there is a scenario in which another server sends a command that should be broadcasted to all (or perhaps select) clients. Does this mean that the application code will have to iterate through all the client connections, pick the relevant (or all of them) clients, and broadcast the message?

This also means that the application code can only exist in one beefy box. Clustering them into several boxes with a load balancer does not seem to be a good idea, as the clients connections will be spread out throughout the boxes. Does this sound correct? How do MMO servers handle thousands active connections?

Suppose there is a scenario in which another server sends a command that should be broadcasted to all (or perhaps select) clients. ... How do MMO servers handle thousands active connections?

MMO games are designed avoid situations where commands would have to be sent to large numbers of clients, let alone everybody. This is not just in their backend infrastructure, but also in the game design. For example, different regions are artificially separated in order to contain the communication burden. Combat heavy areas such as raids can be instanced, which again acts to limit the number of clients that are interacting at once. This feeds into the server design, raids can be load balanced among a pool of servers, each client only needs to communicate with at most one at a time.

There could still be something like an announcement though right, even within a region.

Generally, for each user, there is some management state. Every so often (server tick rate) the game examines the world state, examines what it's told that specific user about, and figures out what to tell the user next. It then makes up a packet of messages, and sends that packet on the socket. Repeat for all users, for all time steps, forever.

If the server has X connected users, and Y steps per second, and Z cores, the max amount of CPU time it can spend on a user is Z/(X*Y) seconds, or it will start to fall behind. Some games can get away with very low values of Y -- 1 or 2, say.

The "world state" for each user could include a queue of messages. A broadcast would then mean creating the broadcast message, and linking it into the outgoing queue of each user. (Ref counting may help save memory here.) However, if a particular player logs in even one simulation step after the world broadcast is sent, that user will not see the broadcast. Thus, it's much better to sync state (and control information) than it is to sync events, for most cases. (Events can still be useful for some things.)

The "world state" for each user could include a queue of messages. A broadcast would then mean creating the broadcast message, and linking it into the outgoing queue of each user. (Ref counting may help save memory here.) However, if a particular player logs in even one simulation step after the world broadcast is sent, that user will not see the broadcast. Thus, it's much better to sync state (and control information) than it is to sync events, for most cases. (Events can still be useful for some things.)

What I'm trying now is having (edit: as in, I'm playing around with. You may want to check into this as well)

1. everything is represented as a "record" (buildings, characters, spells, chat messages, spell casts, etc)

2. records "attach" connected clients by going through the server's client list and filtering (example: is this client in range of me?) at creation

3. clients notify records that it should be attached (records filter this like above) when a client is created or moves a certain distance

4. records go through their attached-client list checking the last updated time stored in this list and update each time a record is changed

(TLDR: records notify the clients instead of clients polling the records for updates)

(I call my objects that encapsulate server-client stuff "server_session"s and the client side has "client_sessions"s which inherit from "session" and servers and clients themselves are "session_processor"s. Basically I try to reuse as much code as possible between the client and server. Whether or not my code compiles into a client or server is determined by a preprocessor #define :p )

Suppose there is a scenario in which another server sends a command that should be broadcasted to all (or perhaps select) clients. Does this mean that the application code will have to iterate through all the client connections, pick the relevant (or all of them) clients, and broadcast the message?

Yes.

Modern games tend to discourage large groups, but in older games you would encounter situations where thousands of players would all be in one location. Anytime these players started casting spells or anything the game world would freeze for a second or two because the server is stuck in that iteration through all clients sending copies of the same exact game world updates to all of the thousands of players.

This is why MMO's switched over to instancing and made group encounters very small. Basically every instance and zone in an MMO is a separate server executable and typically even a separate machine. Of course the downside is that newer games aren't as fun anymore. They feel more like fantasy death-matches than an MMO.

They feel more like fantasy death-matches than an MMO

Because that's what MOST people pay for.

Some other games have been made successfully, though, like the A Tale In The Desert series.

It comes down to server tiering and instancing, as others have mentioned. Each shard could consist of a handful of top-level world servers, and beneath each of them could be multiple region servers, and beneath each one of those many instance servers, etc. Thousands of players may be playing your game at once, but each server would only have a handful of players of connected at once (or not have to do much work per player). You'd also probably have separate servers for logging in, chat, auctioning, finding instances, persisting player data, etc., that serve very specific purposes. Basically, the work is spread out across many servers that are each designed to do one task really well.

Of course, the game design itself would have to be amendable to this sort of stratification. If you're trying to design a game where thousands of players can battle it out at once ala EVE, you'll need a whole separate bag of tricks!

It comes down to server tiering and instancing, as others have mentioned. Each shard could consist of a handful of top-level world servers, and beneath each of them could be multiple region servers, and beneath each one of those many instance servers, etc. Thousands of players may be playing your game at once, but each server would only have a handful of players of connected at once (or not have to do much work per player). You'd also probably have separate servers for logging in, chat, auctioning, finding instances, persisting player data, etc., that serve very specific purposes. Basically, the work is spread out across many servers that are each designed to do one task really well.

Of course, the game design itself would have to be amendable to this sort of stratification. If you're trying to design a game where thousands of players can battle it out at once ala EVE, you'll need a whole separate bag of tricks!

By nature, in an MMO there will be a point in which state must be shared between players in some manner. I help design, build and maintain massively multi-user telco services for work on the magnitude of 100k+ users per cluster. Oddly enough this is far less complex than an MMO :)

As was mentioned, it is always preferable to push out anything not "shared" to a series of load balanced services - i.e. find the smallest thing you can do on a large scale that is not visually time sensitive and do it well, then put it behind load balancers - authentication, chat, UI responses (menus etc.). State always becomes an issue here but many times state isn't as necessary as it seems. Perception is the key concept here and it will be different based on the mechanics of every game but there are some general assumptions that will be made.

Physics and collision detection tend to be things that must be housed in an instance of a specific maximum number of players for example. Proper game design and level/area/zone distribution can do more good than innovative code in these areas many times. Distributing wealth, items, quests across a more vast landscape will allow things to spread out much more than if things are concentrated in certain places. An example of this is "the best x in the game" where "x" is something only found in one spot. If there can be only 1 then the goal of many players will be to acquire this item. If on the other hand, there are many balanced items etc. in the game, players will naturally be divided on which ones they care to quest for the most.

As a service provider, our company has many issues to overcome however uptime/availability and load distribution tend to be the biggest daily engineering feats we manage on a daily basis. I can only imagine that an MMO has similar requirements as they too are ultimately a service provider.

there have a game server frame maybe suitable for u, it use libevent as netlibrary, maybe u can use it as a reference.

my company use it to develop a server and support tens of thounsands of people online at the same time in a gameserver's process.

website:https://github.com/ketoo/NoahGameFrame