How does multiplayer games handle multiple servers? Suppose I have 100 servers from Amazon web services and 100 from Linode. How will I get a list of all the active servers? Should I have to manually make a list of all the servers on a main server and send requests for the server list to this main server? This does not seem very efficient in production.

Some (most?) games work by having each game servers regularly ping the master server list or matchmaker, and it gets the IP from that. This also has the added benefit that servers that go down are removed from the server list, as they've stopped pinging.

The general Google search term you want is “orchestration.”

Typically, there are two levels of orchestration; one is how you provision the physical (or virtual) servers, and one is how those servers then collaborate to provide services (such as gameplay) to the users.

In general, you'll prepare some kind of “bootstrap” image for the servers. This bootstrap will contain some kind of key and some kind of start-up script that lets the servers connect to the central machine management service. This could be a kubernetes kubelet setup script, that provisions nodes, or it could be a terraform client script, that register the machine in a terraform deployment setup, or something else.

The provisioning system will then make sure that the servers have the right versions of software. Often, servers have some tag saying what roles they play ("chat coordinator" or “main game service” or “matchmaker” or “player persistence” or whatnot,) and also some tags that say what domain they belong to ("development," “testing,” “production-north-america-west,” or whatever.) Different versions of the software will go to different domains; there's some kind of configuration that lays all of this out.

Once the servers are available, and running the appropriate version of the software, they will typically “dial home” to some coordinating service. This could be a single machine (which would be bad if it died) or some kind of redundantly-reliable cluster. There are tons of ways of solving this question. The servers register themselves in some registry, and keep sending occasional “I'm alive and have availability X” messages to the matchmaking services, such that they can be assigned appropriate gameplay work/load. An alternative is for the central services to poll the servers to see if they're up, and if they miss some number of polls in a row, they are considered “down” and removed from the available roster.