Hi! 

For my current game project, I'm trying to find a good means of communication between server and clients. I decided that the client will literally only handle the rendering, and will send input over to the server, and the server updates all clients about all changes each tick. 

(This already looks slightly flawed to me since it will be very network heavy I guess) 

This also means that I need to be able to send a lot of different kinds of data over network, and that at the beginning of the game I need to send data about the whole world. 

I'm in the process of trying out two different things, both of which I think are rather ugly and network heavy:

1. Have each of my gameobjects have a serialize and deserialize method. Every tick, I loop through all gameobjects that have been changed in someway and send a packet that consists of the total length of the packet plus all the serialized things in it. Some of the cons of this method includes that I can't differentiate importance of updates; for example, if there's an event that happens rather rarely, I'd want to send it over tcp instead of udp. I can of course split it up and send one packet per entity in my world, but that still seems ugly... 

2. I am writing a reflection based serializer that takes each field that has been marked with a specific annotation and recursively goes through each variable to boil an object representation down to a list of prjmitives. I then send that over network and build it back up. Seems less ugly, but more complicated and "heavier". 

I've never really thought about the details of game network protocols like that before, so I'm quite inexperienced. What would you guys recommend? 

Thanks! 

Quote

 

(This already looks slightly flawed to me since it will be very network heavy I guess) 

 

What do you mean by "network heavy"? What kinds of properties change about your game entities; how many entities are there; and what other mechanism would use much less data?

Quote

 

This also means that I need to be able to send a lot of different kinds of data over network, and that at the beginning of the game I need to send data about the whole world.

 

Yes, all networked games know how to send property updates for their entities, and how to send the initial game conditions when a player joins. For games that don't support late drop-ins, everyone is just told "here's where the players are, and here's the level file you load from disk," for any game that supports late joining, the state of any entity that changed (or spawned, or died) compared to the initial level file will have to be sent on join. No way around that!

In general, game networking engines keep track of which player has seen what changes from what objects, and in every network tick (which is typically between once a second and 100 times a second, depending on game) the game will look at all the un-acknowledged changes for a particular player, select those that are "most urgent," and pack those into a packet that it queues to the player. An update that has been sent, but not yet acknowledged, may be given less urgency for a little bit, but if it remains un-acknowledged, it may pop up in priority again.

Typically, priority for a particular update for a particular player will be based on what that player is close to, who/what that player is interacting with, how big/scary/dangerous the entity is, and so forth, in addition to how long ago the player last got an update for the entity.

If you detect that the player suffers packet loss, you can assume that you are sending too much data, and reduce how often you generate-and-send packets, and also increase the tolerated time between entity updates. You can also introduce a "view distance," where entities too far away aren't shown to the player, and reduce this distance to reduce amount of bandwidth consumed. Similarly, if you detect that there's no packet loss, there may be more network bandwidth available, and if you're not already sending everything you want at the frequency you want, you could turn up the frequency/size a little bit.

There are other models. Most importantly, the "lockstep" model, where players only send inputs, and the input from all players for step X is received on the server before the simulation is allowed to advance to that step. Similarly, each client must run the same simulation with exactly the same (deterministic) math, and will wait until they have the inputs for all players sent to them before they advance simulation. Command latency will be high, and is usually hidden through some kind of "yes, sir!" acknowledgement animation (hence, why this model is most common for real time strategy games, but also works for RPGs, turn-based games, and so forth.)

Late joining (and dropping) in a deterministic game is often quite disruptive, and these games tend to have more lag in cases of those type of events, not to mention all the command latency involved, so this is not necessarily any better or less bad than the other models.

Thank you so much for your answer!

Quote

what do you mean by"network heavy"

Well, since I am doing all of my state calculations on the server, I have to sent all updates to all players - And the updates could vary in since depending on what exactly is happening. In contrary, if I were to run a bigger part of the simulations on the machines of the different players, I could avoid quite a bit of the "network flow", but am having the risk of the state being manipulated or simply being out of sync... So I really don't want to change the approach of running everything on the server, but I'd still like to minimize the data that is being sent. (For example, by taking the "what was updated?" even further and not send the GameObject as a whole over the network once it was updated, but only the fields that have been updated. But I'm not quite sure about how I would code that without making it super messy.)

Quote

"most urgent," and pack those into a packet that it queues to the player.

Makes sense. I'm just not quite sure how to determine priority. I want to keep my game as modular and flexible as possible, which on the other hand means that I barely have any constraints/requirements written-down (I'm just talking about "Game Objects" everywhere, but I don't really know what game objects I'll have), so I'm trying to think of a system that would allow for that.

Quote

 Most importantly, the "lockstep" model

Sounds interesting. I wouldn't use that for the game I am currently writing, since it is rather fast-paced than turn-based, but I'll keep it in mind. 

The way I am handling movement and input in general for now is that I am sending all player inputs as TCP (Considering them all equally important), and for example only sending "Start" and "End" commands for movement - Whereas the server sends back updates every single tick, therefore making potential loss not that important.

Besides protocol/structure, one of the main problems I am currently having is that I am not sure *how* I should send the data back - As in, how do I boil it down to its most important parts and put that into the smallest packets of bytes possible? This is more about the implementation than the concept I guess, but I was wondering how it is usually done. (Currently, I am using a system where each gameobject handles its own (de)serialization, but it's kind of ugly)

if I were to run a bigger part of the simulations on the machines of the different players, I could avoid quite a bit of the "network flow"

You need to send a flow of state updates anyway, because you need to put the clients back in sync when they de-sync, even if they simulate. Only in the lock-step deterministic model do you get away with sending less. The only question is how much / how often you send the state updates for each entity, and how big the entity state is.

Regarding talking about "game objects," that's fine as far as it goes, but you will also need to talk about "properties" of those objects, and "serialization" for those properties, and "change notifications" (or "diffs") of those properties of those objects. Once you have that, you can build a reasonably efficient representation of what actually updates.

It also matters whether you use TCP or UDP. If you use TCP, you can assume that the player has seen everything you send, although sometimes with a longer delay. If you use UDP, you can assume that players will see updates reasonably quickly, but may miss some of those updates, so you need some kind of acknowledge (and perhaps not-acknowledge) to tell the sender what the receiver actually got and didn't get.