Hello,

I have a web-based game platform that relies on WebSockets. In this game, up to 10 gamers can join a match together (but not necessarily at the same time. Some can join first, and others can join later) to play a game against each other. There can be more than tens of thousands of players playing the game at the same time, but only 10 players max per match/room.

I have been using a default load balancer service provided by my cloud platform which distributes user connection to one of many available HTTP servers in the backend. When 10 users, who are in the same room, end up in different backend HTTP servers, the servers communicate with each other using Messaging platform (Something similar to Apache Kafka) to relay and communicate actions happening within a room.

Things are working fine, but this approach has few issues.

• Messaging platform can become a bottleneck when we scale. (It gets worse as we increase # of HTTP servers)
• Messaging platform adds latency in edge cases.
• Messaging platform adds cost.

Therefore, I am looking for a solution to see if I can somehow route users who are supposed to be in the same room to the same backend game server in one go. If all users/players from the same room are connected to the same server, there is no more need for an inter-server messaging platform.

Of course, I could potentially separate my current server which is doing both HTTP handling & Game logic into two sets of servers, one set for Frontend HTTP Servers and the other set for Game Servers like below.

From

x WebSocket Clients ←--→ y (HTTP & Game) Servers ~~~~ MessageQueue platform

To

x WebSocket Clients ←--→ y HTTP Servers ←--→ z Game Servers

With the latter approach, I could have more fine control of maintaining a set of game servers.. but it will require some substantial dev effort on my end to change current architecture like that. Additionally, the assignment of room/matches to the z Game servers is still a problem to solve.

Therefore, I am looking for a way to solve this problem without making substantial changes to the system if possible. (Perhaps, with the use of some smart reverse-proxy load-balancer or framework in front of HTTP servers?)

My requirements would be something like

1. If no player has joined for a room or match A1 yet, then the first player to join can end up in any one of the backend HTTP servers.
2. When there is at least one player already connected to backend server S1 for match A1, some mechanism should forward later joining players to server S1.
3. The number of backend servers up and running can change any time before/during/after when 1 and 2 are happening.

If anyone has insights on this or advice to give… please let me know.

I highly recommend separating “playing a game in a room” versus “waiting in a lobby to join a match.”

While waiting in a lobby, you can use whatever hash partitioned approach you want. If you have chat and movement, perhaps send it through the messaging system – this will not be the bulk of your load.

Once you know which players to to a particular room, allocate the room to a server machine (container host node, or whatever your system uses,) and give it a unique address of some sort. Exactly how you implement this depends on technologies available.

For example, each container on a node could have a separate listen port. Or each instance could use a separate virtual network interface address.

The LB then has to be configured to route the user to the appropriate target host:port. This can be done either by making the LB route based on HTTP path or you can pre-register many thousand DNS names, and map them to the specific instances. For path, you'd make “instance.yourgame.com/node-01/port-3000” route to node 01, port 3000; “instance.yourgame.com/node-03/port-2820” would route to node 03, port 2820, and so on. For DNS, register “game-01-3000.instance.yourgame.com” to go to node 1, port 3000; register “game-01-3001.instance.yourgame.com” to go to node 1, port 3001; register “game-04-5811.instance.yourgame.com” to go to node 4, port 5811 and so forth. Typically all the game node DNS names would map to the same IP address, but the LB would use the Host: header to know where to send the traffic.

The load balancers will need to know either about the route to node/port mapping, or the DNS-name-to-node/port mapping. In typical load-balancing solutions, you set this up using direct “front-end” to “back-end" mappings, and you'll probably want to write a script to generate those configuration files to stay sane.

You will note that this has the unfortunate requirement of actually keeping track of which hosts and ports are available on the inside, to allocate games to. There's really no way around that – if you want to do dedicated player allocation, you need to have something that knows how to allocate and route those. Maybe have a controller process on each node that reports how many containers are alive, and is responsible for allocating/managing free ports for new instances, and have those report back to the lobbying system.

If your hosting provider isn't cool with (tens of) thousands of separate front/back-end mappings, pre-allocated to the hosts/ports you will be using, then alternative is to have a single load balancer, and have that balance to one or a few machines, whose only role is to be game traffic routers. Those machines would in turn keep the mapping of “room” to “host node and port” and forward the connections as appropriate. As long as multiple machines have the same mapping, the load balancers can spread the load across them, for some amount of redundancy. You basically move the proxy/LB job into your own process/es at this point.

@hplus0603 Thank you so much for the guidance. I think what you said will likely work for my case.

if you want to do dedicated player allocation, you need to have something that knows how to allocate and route those

I was kind of hoping for shortcuts, but that does make sense. At least, it will be a fun problem to solve.

So in summary, if I use a path-based routing approach over DNS, I will have to do… (I am running backend in Kubernetes)

1. Build a room tracker service to keep track of (room, serverIP:port) mappings.
2. Learn & figure out dynamically updating path-based routing for reverse proxy at runtime.
3. Use path-based routing to direct/relay users to the correct serverIP:port.
4. Figure out a good way to discover newly added hosts in Kubernetes when scaling up servers. (or vice versa when scaling down or replacing hosts)

I think the trickiest part for me is #4, and #1,#2,#3 all depend on #4, but I will do some research and see how it goes.

P.S. I really wish I can take a look into Fortnite's backend source code someday to understand how they handle 100 users per match for millions of concurrent users.

It's very likely they don't use Kubernetes. I like kube as much as the next guy for web-based services, but Fortnite uses Unreal networking, not HTTP or websockets. They very likely use the “allocate users to server instances, run X server instances per host” approach, because that's the most effective/efficient.

You could even spin up servers where each one has a public IP, and just run a few instances (on different ports) per server instance, and just point players straight at the public IP. This is what Roblox used to do (haven't checked lately; they may have changed that when they moved servers from Windows to Linux, but if it works, why change it?)