Hey guys, we're doing an FPS with user-hostable servers, and have found that most of our testers (ourselves included) get horrible throttling from our ISPs if too many players come in at once. At about 10 players, pings will spike from < 100 to oftentimes as high as 15000. A lucky few have much more liberal internet providers, and when they host we can pile in and push through endless physics events, network events, etc. with as many players as we want- no problem.

How is this being solved these days? Any good workarounds?

Much appreciated,

ZA

I'm guessing that the server's total uploads per second is more than the capabilities of your users... at which point packets start filling up a queue faster than they're being sent, which causes 99% packet loss to start occurring, which you're detecting as massive ping times.

You need to either tell your users how many players they can support on their upload bandwidth, and/or optimize your game to use less bandwidth.

Most residential DSL type plans will have ~10x more download bandwidth than upload bandwidth -- e.g. 20Mb/s down and 1Mb/s up, or 5Mb/s down and 0.25Mb/s up.

This kind of connection is fine when you're acting as a client, but is not ideal for a server.

1Mb/s is equal to ~122KiB/s. So if a user only has a connection with 1Mb/s of upload bandwidth, and you want them to be able to host 30 players, then you need to design your game so that each client only requires the server to send them <4KiB/s of data.

Most games that are designed for residental user hosting, simply just don't support 30 players...

Diagnose the problem:

* Ask your users what speeds they're promised from their ISPs.

* Ask them to use a site like http://www.speedtest.net/ to test their actual upload speeds.

* Add code to your game to measure the amount of data that you're sending in each direction per second.

You should be able to come up with some guidelines for hosting -- e.g. the server requires 3KiB/s of upload bandwidth per player.

Also, if you add this measuring code so you know how much network traffic you're generating, you might find that certain systems are using a disproportionate amount of data, and that you've got some good targets for optimisation work...

What kind of network synchronisation model are you using for your FPS? Have you based it on another FPS game's model, like Unreal, Half-Life, Quake 3, etc? How often do you send out updates, etc? Can you tweak this, so that people with worse connections can update their clients at 15Hz, while better servers can update at 30Hz, etc?

First: As Hodgman said, are you sure that the problem is ISP throttling?

Network connections have limited capacity. Users may run BitTorrent, stream Netflix, have automatic backups, get Windows Updates, or be signed up for IPTV-based "cable" TV. Your game may generate more traffic per event than strictly necessary.

If you know that the problem is ISP throttling, then tell users to get a better ISP :-) There's a reason places like serverbeach.com rent out game servers by the month.

If not, then you should probably make sure to make your game use as little bandwidth as possible per event.

Also remember that the incoming network traffic to the host scales linearly with the number of users, but the outgoing network traffic from the host scales geometrically.

Assume for the sake of argument that there is a host-local player playing.

Let's look at the values for (Number of Players / Incoming Network Traffic / Outgoing Network Traffic). For this, x will be the data required to be sent for the non-player actors in the game space (any game state changes, bots, NPCs, terrain deformation, whatever), and p will be the network traffic required to update the actions of the average player controlled actor (moving about, shooting, taking damage, etc).

At 2 players (1 local, 1 remote), we receive one players worth of actions, and we send one players worth of actions (the server player, sent to the remote player) plus the NPC traffic. So inbound is p, outbound is p + x.

Now go to 3 players (1 local, 2 remote). We receive 2 * p traffic, but our traffic just ballooned. Now we are sending 2 * p traffic (since we have three players in the game), but we are sending it to both remote players. So our outbound traffic just went from p + x to 4p + 2x.

At 4 players, we receive 3 * p traffic, but we're sending 9p + 3x traffic. Our outbound traffic is effectively defined as (NumPlayers-1)^2 * p + (NumPlayers-1) * x.

This doesn't sound like ISP throttling, this is geometric network traffic increases taking out your server upload.