Advertisement

MMOs and modern scaling techniques

Started by June 10, 2014 01:26 PM
65 comments, last by wodinoneeye 10 years, 4 months ago

In my opinion, gameplay comes first, then we imagine software to make it possible. I mean that building an angine to enable compact crowds of players with collision & physics simulation don't looks like enough to define the player experience and what really matters. Is the compact crowd (public transportation like) even something desirable to provide a good experience? If not, the issue should be dealt with before (disable collisions in crowded/marketplace areas)

And if you need that in the game (large scale -interesting- close combat battles?), I doubt that a generic engine would do.

That mean a lot of testing & tweaking, keeping in mind that what's important is not accurancy but player experience according to the gameplay mechanics. Players everywhere would mean that individuals loose significance. That's not PvP, but Mob vs Mob. Maybe some sort of LOD with precise local interactions then working with densities and statistical behaviors at longer range to reduce the quantity of informations required. In this case you have to deal with the issues listed by hplus0603. At the end it depends on the game's priorities and compromises.

PS: I have met web architects advising to use simple stateless loadbalancing for realtime multiplayer with Redis to manage send queues and game state. If you need physics or whatever, it won't work. Too much latency, and that's not Redis fault. The software is fine but not made for this. In my opinion, that's shoe horning REST optimisation into something which cannot be made stateless. Because there are to many player interactions. It works for chats however because: 1) There is not so much user interactions 2) latency is not really a concern.

In my opinion, gameplay comes first, then we imagine software to make it possible.


More than one game company has died by designing something that, in the end, the engineering team couldn't actually deliver.
enum Bool { True, False, FileNotFound };
Advertisement

In my opinion, gameplay comes first, then we imagine software to make it possible.


More than one game company has died by designing something that, in the end, the engineering team couldn't actually deliver.


Death by Romero. Or Blackley if your prefer.

While gameplay and story definitely trumps graphics, designers need to know what tech limits they have and work within (and press against and stretch) those limits as far as possible, not dream up whatever amazing thing comes to their mind, and then realize 5 months before the deadline that you'll have to make major cuts to actually be able to release the game on current technology. Having to cut alot of gameplay features is like poorly resizing an image and leaving ugly graphical artifacts. It ruins the cohesiveness and polish of the game.

In my opinion, gameplay comes first, then we imagine software to make it possible.


More than one game company has died by designing something that, in the end, the engineering team couldn't actually deliver.

Fine, I cannot disagree with that smile.png But that's not the what I meant: Many companies died by designing something which was way more complicated and beautiful than what the actual need required. So without clear goal & specifications, we (myself first) tend to overthink things.

Sorry to have implied that. At the end this question is more a question of well designed development process, and evaluation. And thanks for the reference, I didn't know it.

More in line with the topic, a few years ago I had read from Bigworld about using simulated players to evaluate the scaling behavior of MMOs. With the availability of AWS or Azure, do you know if this kind of testing has been used effectively by studios? Apart from the difficulty of simulating pertinent player behaviors, are there other pitfalls?

More in line with the topic, a few years ago I had read from Bigworld about using simulated players to evaluate the scaling behavior of MMOs. With the availability of AWS or Azure, do you know if this kind of testing has been used effectively by studios? Apart from the difficulty of simulating pertinent player behaviors, are there other pitfalls?

I don't know about MMOs, but the indie game SpyParty (a 1 vs 1 game) wrote up a series of articles about using Amazon to loadtest their lobby and game hosting server:

Loadtesting for Open beta, Part 1 & [Part 2], [Part 3], [Part 4]

I found it to be an interesting read.

Fascinating subject. A couple of observations and half-remembered stories, which I hope contribute a little bit, even if they don't answer Ben's original questions:

- A good friend of mine worked on a proper MMO back in 2001 (Rhyzome), and I remember him doing the math for me and showing that he had, I dunno, 10 CPU cycles to decide which data to send where (based on the data fields per player and the number of players per server). So he had to write multiple levels of prioritization algorithms. He told me this story to illustrate how unsuited big, expensive web servers were for MMOs. This has probably changed, although I don't know to which degree.

- Unlike what Ben said I am seeing signs of splitting things up into multiple servers per service type in the web world. It seems to have become the architectural style du jour, in fact. But I have no strong evidence, this is just what I'm picking up.

- State seems to be the key difference. HTTP is stateless, games not so much. The rising popularity of unit testing and test-driven development correlates with the rise of web dev. The same goes for functional programming and the rise of back end languages like Erlang and Clojure.

- I've seen a case in a social game company of engineering pushing back design because of increased state. I assume this happens a lot. I've certainly seen enough cases of eventual consistency in social games.

- "Proper" MMOs are probably the cheetahs (or koalas) of the engineering world. They grew for the historical reasons Ben described, and now they occupy a niche that is very hard to fill in any other way than the way they do it.

- If you want to know how the big web companies do scalability, I highly recommend http://highscalability.com There's a lot of material (presentations, papers) available. Favorite arcane detail: bidding algorithms to control EC2 costs...

Advertisement

Higher complexity can require scaling to be even greater (not linear) than previous simpler games.

(Good) NPC AI for example with farmed-out AI processing -- and those seperate NPC AI computers having to maintain their own local world representations - with all the volumes of world map updates flowing (hopefully across a high speed server network). Now with the greatly increase data traffic through individual world map zone servers (the state book keeping process) that starts overwhelming/burdening them (requiring yet more of them and the overhead of the zone/area edge handling - for large continuous worlds)

Communication bound limitations as a secondary effect to the data processing bound (and AI uses magnitudes more CPU and significantly more local data per 'smart' object)

The new complexity can require another O(N^2) expansion as there are more interactions across CPUs handling fewer and fewer objects each (and the traffic having to go across the much slower network interface instead of within the same shared memory space)

--------------------------------------------[size="1"]Ratings are Opinion, not Fact

Ping times are a source of complexity and gameplay challenges, but they are not a source of scalability problems.

Ping times for wired connections will not drop dramatically in the future, because they are bound by the speed of light -- current internet is already within a factor of 50% of the speed of light, so the maximum possible gains are quite well bounded.


I'm not sure if I'm misreading you. But I feel like what you said is very misleading. The real world performance of network infrastructure is not even slightly approaching 50% light. We typically max at 20% in best case scenarios.

The majority of transit time is eaten up by protocol encoding/decoding in hardware, and improving the hardware or the protocol can dramatically increase transit latency. Ex. Going from tcp to infinband inside a cluster can reduce latency from 2milliseconds to nanoseconds.

Not saying it's practical by any means, but we're bound by switches/protocols far more than light.

And by the number of 'hops' along the way of the path the data takes (repeating the above overhead over and over).

Maybe in the future we will have a more 'Johnny Canal' (https://screen.yahoo.com/johnny-canal-000000884.html) type Internet system

with fewer hops but well that costs lots of cash ...

--------------------------------------------[size="1"]Ratings are Opinion, not Fact

This topic is closed to new replies.

Advertisement