Advertisement

(Managed) Master Server - hardware requirements and cost?

Started by January 09, 2017 12:09 AM
9 comments, last by hplus0603 7 years, 10 months ago

Another one of my questions regarding server hosting (hope i don't annoy anyone here. :P)

To give you a bit of insight before i ask the question:

I'm currently working on a platformer/racing game.

The idea is that players can race on maps and win medals (depending on how fast their run was). Custom maps can be created to by using the built-in map editor.

Now, naturally to get the most out of the experience i wanted to create a master-server which includes features like:

  • online leaderboards
  • statistic calculations (ladderpoints, current medal count, etc... NOTE: I don't run game-simulation on the server.)
  • custom map sharing (players can log into the website and upload their custom maps which in turn allows others to download those from the web)
  • 3D realtime WebGL based previews of said maps (you can preview maps in realtime in the webbrowser before downloading them
  • server list for hosted game-servers by the players. (Hosted servers by the community register themselves on the master server so that others can see this particular server in the serverlist.)

The server could simultaneously be used to host other stuff like a forum and/or a blog for the game.

Now the big issue is that i have absolutely no idea how demanding this stuff will be in terms of hardware requirements and/or bandwidth (and traffic). Im quite limited when it comes to the budget which i can invest into the server per month. (It may be my first ever commercial release.)

I know that the general answer to stuff like this is "benchmark it!". But i hope that some of you with experience on this topic can give me advice/some tips to get an general idea on how beefy a server has to be in order to handle such a load.

Here are some details about how the communication between the client and server works and how some of the mentioned features were implemented on the serverside:

  • Thew whole server-client communication is done via HTTP requests.
  • Leaderboard requests are simple HTTP requests to a PHP script which reads the top 10 entries of a particular leaderboard + the players individual position. (around 2 to max 3 select statements to the database.).
  • After each race, the client sends data (like palyerID, score, + some minor metadata) to the server and retrieves the leaderboard after the requests. (so those are 2 HTTP requests after each run.) Depending on the players skill, each map can be finished in a timespan of between 30 seconds and 2-3 minutes. (this is the average interval in which those requests could happen.)
  • Statistics are calculated by using Cronjobs which run in an interval of 1-2 minutes on the server. In order to avoid heavy calculations, i update only 20-50 players per interval (want to save on CPU performance as the calculations can be quite heavy.).
  • Mapfiles which can be uploaded to the server are rather small. Average size is between 32kb and 64kb (for middle to larger maps). In order to save space i compress those files on the serverside with gzip to reduce the filesize. (I also parse them on the serverside to make sure that this file really is a valid gamemap file.)
  • The 3D WebGL based preview in the webbrowser downloads the selected map from the server as a compressed gzip file (to save traffic), extracts it on the clientside with javascript and then parses the file.
  • Server list would also be updated with plain HTTP requests. A community hosted server could (as an example) send a "i'm still alive and have the IP XXX" HTTP request to the server every 1-3 minutes and insert/update this information in a mysql table. (Although i'm not sure if that would be feasible/a good idea for the server performance.)

Another thing that is worrying for me is running an unmanaged VPS.

I've done quite a bit of research on this topic and came to the conclusion that i have no idea how to setup,secure and maintain an unmanaged server. (I don't have the time and knowledge to do that.)

So i was looking into a managed server solution, though those are less common and more expensive compared to unmanaged solutions.

Now the question is, given that the communication is done by simple HTTP requests, i could technically use a simple managed Webserver/Webhosting solution (instead of an unmanaged VPS) so that i don't have to worry about configuring the server itself?

As an example, i have found this company (seems to be based in germany, although i have read that their datacenter is in france> https://www.hosteurope.de/en/WebServer/Comparison/ )

The 20 euro solution has 2GB quaranteed ram, unlimited MySQL databases on the HDD, 2 MySQL databases on a SSD (which would be ideal for the games data),100GB HDD storage (more than enough to store a ton of custom maps) and a "traffic flatrate", meaning that i have "theorethically" unlimited traffic. (Although they seem to have the right to limit the bandwidth of the server from 100Mbit down to 50Mbit if i start to stress the connection too much over a longer period of time.) The issue here is that they don't seem to specify how many cores are 'quaranteed" for this virtual webserver. (Probably none?)

My question now is: What kind of server (in terms of hardware like CPU and RAM) as well as traffic and bandwidth would be required in order to host a server like this? Especially if you have a bigger number of concurrent players (like 1000? Is this realistic?)

I don't want specifics, only a rough estimate in order to get an idea on how demanding that stuff can be. (Some of you may have done something similar before.)

If you have suggestions for companys which offer cheap services which could handle stuff like that i would love to check them out. :)

Of course, if it turns out that running such a server would be way too costly, then i would simply drop the whole masterserver altogether.

Here's my advice:

Stay away from VPS if you want any modicum of performance.

Sure, VPS is cheap, and you seem to get a lot of 'gauranteed' ram etc for your money, but the big problem with VPS is IOPs, or I/O instructions per second.

For most VPS solutions, this is quite low, so compared to renting actual physical hardware, you get very poor disk performance which manifests in slow DB queries, slow file access and copying etc.

Ive had lots of experience in both VPS and dedicated/co-located solutions, so i can speak based on what i've had to deal with.

My advice is to rent a dedicated server, even if you have to rent it slightly further away, sure the ping times will be a little larger, but you're gauranteed all the resources of the machine. If you really want to, you can split this machine up yourself using (for example) vmware esxi, which can be used for free. Even then, your performance will be better than a "cloud hosting" solution, unless you throw thousands of dollars at it.

For reference, i pay $80 CAD for a server in canada, hosted near Toronto, which has always been very good for me and the routes are great from there to the UK so i don't have issues with latency. Of course, if i'd got a server in london at Telehouse, it would have been faster still, but I'd pay the premium for it.

I don't think you need that much in terms of ram though, or disk space. I have 16gb of ram and a 300gb disk, which is far more than i ever use.

I host a chat network on my server which about ten years ago hosted a live developer chat for Blizzard's World of Warcraft, and this is back when i only had 4gb of ram. That evening, over 8000 people were concurrently connected to my server, so i don't see the need for lots of resources unless you suddenly become the next major web based game, in which case... your software will likely be the bottleneck first!

if you need to know more just reply here, i'd be glad to offer more help.

Hope this is useful to you! :D

Advertisement

A dedicated server would be the best solution performance wise, but it is (for me at least) waaaay to costly.

(I'm searching for a managed solution that would cost me around 20 euros per month. 30 euros is the maximum i'm willing to pay. Anything above that is way to risky when it comes to finances.

Though i realise that this might not be possible in this price range. That's why i ask here for advice.)

When it comes to I/O performance: Is it really that bad on a virtual server? Latency is not the primary priority (i'm not running a game simulation on the server after all.), as long as the HTTP requests are processed in an acceptable timespan it should be ok. (Having a server with a SSD should also mitigate this issue somewhat. The company i mentioned above does allow you to have up to 2 MySQL databases on an SSD which would increase the SQL performance immensely.)

The primary concern for me is that the server hardware (RAM,CPU, I/O performance, etc...) on a low cost (managed/web-) server would be too underpowered to handle that load. (I don't want to find out after the games release that the server is way too underpowered and would require a significant upgrade in order to run properly performance wise. (The server provider might also step in and say that i'm overloading their server.) This would eat into my budget and would in a worst case scenario force me to shut down the servers altogether as they wouldn't be financially viable in a long term scenario. Not really great if the game is sold commercially.)

If you're releasing commercially, don't skimp on servers.

You'll only find yourself having to upgrade them later and it will cost you lost customers then, as players get angry at lag and slow response times and go elsewhere...

Spec out a server that covers the worst case scenario.

Given that the game doesn't run the simulation on the server, I don't think you need dedicated hardware. The sort of latency and throughput requirements you have are not significantly different from commodity website or API hosting, so you could use a 'backend as a service' provider for most or even all of this. It could be a general-purpose system like Firebase or perhaps a more game-focused approach like Photon, but I strongly recommend you investigate how far you could get with these before committing to developing and maintaining your own servers and server software. I doubly recommend this given that you don't have much expertise in this area, and what you potentially save on not paying for a service you'll lose on development time, bugs, and potentially security risks.

If you're OK with your own system administration, it's hard to beat Amazon Lightsail for value-for-money in hosting.
The servers are solid, the data centers are first class, and the included networking transfer budget is enough to run your typical game master server without worrying about overage charges.
And you can get started for $5/month! For a small-to-medium-size game, the $5 server may be all that you need for the game master server for a long time.
Lightsail is like Amazon Elastic Compute Cloud, except with a little less flexibility, and a lot less cost on the low end. If you find that you have a massively successful game on your hands, migrating to EC2 from Lightsail is very simple.
enum Bool { True, False, FileNotFound };
Advertisement
Stay away from VPS if you want any modicum of performance

Second that, although I wouldn't say IOPS is the problem. Jitter is, in my opinion. Absolutely unpredictable scheduling jitter. Works perfectly well 95% of the time as if you had a dedicated server. The difference is not in any way noticeable if you run e.g. a webserver. On the other hand, it totally sucks ass for anything "game" or "game related".

Note that Hosteurope is pretty much the absolutely, undisputedly, most expensive hoster in Germany (at least, used to be... have not looked at prices for 3-4 years). As in, about twice as expensive as the more expensive ones, and three times as much as average. :lol:

A dedicated server needs not be all that expensive, it's quite affordable these days. Really, do you need 32 cores and 128GB of memory? What computional tasks on hundred thousands of concurrent players are you planning to do?

Don't you think that with a "reasonable" workload a quadcore with 8-16GB will -- with ease -- completely saturate the ethernet link? You get that hardware from quality hosters for around 49 per month, no bandwidth quota. Or 69 per month if you think you need 6 cores and 32GB.

Just as a reminder (as it seems that some of you think that i want to host a server which runs game logic.)

I don't need to run a true "game" server which handles the game simulation.

What i'm talking about is basically a HTTP leaderboard server with a few extras thrown in. (Like statistic calculations via cronjobs, hosting of custom player content (small 32KB map files), etc...)

The clients don't have an "active" socket connection to the server. They only send HTTP requests if they have to. Other than that, the game client doesn't depend on the server at all.

This server would also be used to run the website as well as a forum (and maybe a blog).

The question is how beefy such a server has to be in order to be able to cope with (lets say) 1000-3000 clients (is this even realistic for an indie game to have that many clients online at the same time? XD) which during their playtime send HTTP requests to either update their leaderboard score and/or retrieve the current leaderboard of the game. (In the original post i roughly explained all the stuff the server has to handle.)

If you're releasing commercially, don't skimp on servers.

You'll only find yourself having to upgrade them later and it will cost you lost customers then, as players get angry at lag and slow response times and go elsewhere...

Spec out a server that covers the worst case scenario.

Well, the question how much power such a server would need for a worst-case scenario.
I have no experience how CPU speed/core count and the amount of RAM would impact a server.

Given that the game doesn't run the simulation on the server, I don't think you need dedicated hardware. The sort of latency and throughput requirements you have are not significantly different from commodity website or API hosting, so you could use a 'backend as a service' provider for most or even all of this. It could be a general-purpose system like Firebase or perhaps a more game-focused approach like Photon, but I strongly recommend you investigate how far you could get with these before committing to developing and maintaining your own servers and server software. I doubly recommend this given that you don't have much expertise in this area, and what you potentially save on not paying for a service you'll lose on development time, bugs, and potentially security risks.

Exactly. The server exposes a few PHP scripts which act as some sort of API for the game which then can be accessed by sending HTTP request from the client to the server. I just don't know how demanding this stuff can be if 1000 concurrent game clients send HTTP request to the leaderboard every time they want to retrieve or update the leaderboard.
How much RAM or CPU power is needed? Is this too much for a "simple" webserver (like the one from hosteurope which i linked in the first post?)
I simply lack the experience in the production enviroment of webdevelopment in order to estimate the hardware requirements of any serversoftware.
That's why i'm asking you guys. (Maybe some of you hosted a leaderboardserver with a forum and a website on a single server? If so, how well did it run on the particular hardware you were using? How many players was it able to handle, etc...)
As to the server development: I can write PHP code and gained some experience in that field. (School and hobby projects. Never worked for a company though. I'm still a student. :) ) So the software isn't really a big deal. (Most of the stuff already works. I'm just not sure if running this stuff will be viable in a production enviroment.)
However, i have absolutely no experience in Linux and less so in working/maintaining a Linux server. But i would be able to work with some form of webhosting solution which simply gives me a capable HTTP server and FTP access.

If you're OK with your own system administration, it's hard to beat Amazon Lightsail for value-for-money in hosting.
The servers are solid, the data centers are first class, and the included networking transfer budget is enough to run your typical game master server without worrying about overage charges.
And you can get started for $5/month! For a small-to-medium-size game, the $5 server may be all that you need for the game master server for a long time.
Lightsail is like Amazon Elastic Compute Cloud, except with a little less flexibility, and a lot less cost on the low end. If you find that you have a massively successful game on your hands, migrating to EC2 from Lightsail is very simple.

Well, the issue is that i have zero experience in working with linux server administration. And after a bit of research i came to the conclusion that managing such a server is way too much hassle for me. (Updates, maintenance, security,etc...)
Though, what do you mean with "small to medium sized game"? The 5$ server has just 1 vcore and 512MB ram. What exactly is that able to handle?

Stay away from VPS if you want any modicum of performance

Second that, although I wouldn't say IOPS is the problem. Jitter is, in my opinion. Absolutely unpredictable scheduling jitter. Works perfectly well 95% of the time as if you had a dedicated server. The difference is not in any way noticeable if you run e.g. a webserver. On the other hand, it totally sucks ass for anything "game" or "game related".

Here is the thing: I'm not running any game simulation on the server. So jitter isn't really a concern for me. The game sends HTTP requests to the apache webserver if nessecary to update the highscore or retrieve some other information. (Like player statistics, leaderboards, etc...)

As i said, the whole server would be able to run entirely on a plain simple HTTP server, as long as it is capable enough to keep up with the amount of HTTP requests (Including Database reads/writes). (The question is how capable the server has to be.)

Note that Hosteurope is pretty much the absolutely, undisputedly, most expensive hoster in Germany (at least, used to be... have not looked at prices for 3-4 years). As in, about twice as expensive as the more expensive ones, and three times as much as average. :lol:

A dedicated server needs not be all that expensive, it's quite affordable these days. Really, do you need 32 cores and 128GB of memory? What computional tasks on hundred thousands of concurrent players are you planning to do?

Don't you think that with a "reasonable" workload a quadcore with 8-16GB will -- with ease -- completely saturate the ethernet link? You get that hardware from quality hosters for around 49 per month, no bandwidth quota. Or 69 per month if you think you need 6 cores and 32GB.

I was looking at hosteurope as i have heard quite a lot of good things on the internet about them. (Support, etc...) And they have a lot of different services to offer. (webhosting, webserver, webserver dedicated, V-server, dedicated server, etc...) But you are right. They are in some cases more expensive to other companys.

I'm open for alternatives though.

1000-3000 concurrent http connections is a LOT.

If you had this many you'd need a significant amount of ram, bear in mind if each of these calls php, and you're using fpm (recommended, don't use mod_php) you'll need enough ram for the php context/interpreter for each of these concurrent requests.

This very soon becomes many gigabytes...

The difference is not in any way noticeable if you run e.g. a webserver.


Depending on the vps, I beg to differ.

I've had a website hosted on a pretty solid vps with ssds, and it was running magneto, which is a pretty popular open source shop software written in php and using MySQL.

It was all well and good until someone (Google) crawled it and made 1000 concurrent Web requests, it crippled the entire vm so bad I couldnt even log in and had to reboot it.

Even just in a Web scenario, many vps are fragile and targeted at development rather than live deployment.

Note I don't bundle amazon and Microsoft Azure etc into this basket, they're a different kettle of fish, so long as you're prepared to pay for the resources...

This topic is closed to new replies.

Advertisement