I'm working on a server-authoritative multiplayer game that procedurally generates large levels with a square grid that represents the level's walkable space.  The grid is fairly high resolution, it uses up about 20-40mb of memory per level.  My concern is scalability, if I have 500 players on a single machine in their own unique levels that's ~20gb of square grids the server has to keep around in memory.  Is this going to raise any concerns that I should be thinking about right now?

45 minutes ago, Crayz92 said:

The grid is fairly high resolution

You should consider LODs if it's 3D. For 2D you can remove the ones not visible to a player.

47 minutes ago, Crayz92 said:

if I have 500 players on a single machine in their own unique levels that's ~20gb of square grids the server has to keep around in memory.

20 - 40 mb of memory for a floor tells me your going to have other problems long before you hit the 20gb mark. What engine are you using?

Have you considered instancing, https://en.wikipedia.org/wiki/Geometry_instancing, this is what it was made for.

1 minute ago, Scouting Ninja said:

You should consider LODs if it's 3D. For 2D you can remove the ones not visible to a player.

 

20 - 40 mb of memory for a floor tells me your going to have other problems long before you hit the 20gb mark. What engine are you using?

Have you considered instancing, https://en.wikipedia.org/wiki/Geometry_instancing, this is what it was made for.

Apologies, by grid I meant strictly in an object/data sense stored as a 2 dimensional array, no geometry, meshes, or rendering is at play. 

Why do you need a copy of the level of the server, if each player is in their own personal world?

Also, if you make a level that's that big, do you think a player will actually manage to walk all over that level? Or could you incrementally generate blocks of the level as the player gets closer?

Let's assume your world is 2000x2000 cells, with 10 bytes of cell attribute data. (Let's also assume you use value structures, not reference/heap objects, else you'll have massive memory waste!) If you generate 50x50 blocks around where the player currently is, you only need to instantiate 150x150 cells (nine blocks of 50x50) at one time. When the player moves out of range of a block, you could even unload that block, and re-generate it on demand, if needed. Or you could save it -- presumably players can't actually visit most parts of a map in a reasonable-length play session?

19 minutes ago, hplus0603 said:

Why do you need a copy of the level of the server, if each player is in their own personal world?

Also, if you make a level that's that big, do you think a player will actually manage to walk all over that level? Or could you incrementally generate blocks of the level as the player gets closer?

Let's assume your world is 2000x2000 cells, with 10 bytes of cell attribute data. (Let's also assume you use value structures, not reference/heap objects, else you'll have massive memory waste!) If you generate 50x50 blocks around where the player currently is, you only need to instantiate 150x150 cells (nine blocks of 50x50) at one time. When the player moves out of range of a block, you could even unload that block, and re-generate it on demand, if needed. Or you could save it -- presumably players can't actually visit most parts of a map in a reasonable-length play session?

Well players can be in each-others worlds as well, and monster pathfinding and whatnot is handled by the server.  As far as the dimensions, levels are somewhat linear so it takes up a lot of extra area.  Here's an example:

So I stored the grid as value types and it's 1/10th the size it was before.  The map in this image is roughly 2048x2048 and 4mb.  I'll just have to rewrite my pathfinding to work with values rather than objects

Okay, so that was an easy win :-) Now, look up multi-resolution grids, or perhaps quad trees, and you may be able to only store the bits of the map you really need :-)