Hi, I'm facing the following question while working on my networked game. The server-side logic is something like this:

1. Input Received by the Server and Enqueued (stamped by the client with a sequence number)
2. Each Physics process (60hz), an input is taken from the queue and processed (if the queue is empty, repeat the last input)
3. Each Network tick (20hz) a game state snapshot it sent to all clients, as well as the last input stamp processed for them

This works fine, the clients then take the game snapshot and do interpolation, CSP and different things. It works.

Then why would I want to sync clocks? Why would anyone want to do it? I understand that there is a method of doing game networking where the “client is in the future”; it sends inputs to the server stamped with the Tick it should have when arriving to the server, so if t=100 in the server the client will send something like t=150 assuming it will take 50ms to reach the server, then the server at t=150 will process all correspondant inputs, that requires having synced clocks, but why do that when you can just process what you've received?

Thanks!

razcodes said:
This works fine, the clients then take the game snapshot and do interpolation, CSP and different things. It works.

It will depend on your game, but it does trigger skepticism. Have you tried this under bad network conditions? High latency, high traffic, and error-heavy network conditions tend to break this down badly. I've seen a few games where people thought it was fine, but their testing was only on their local network.

Running physics across a network is an amazing way to destroy network performance, as values are updated and sent back over the wire. One I was brought in to help with a game that had implemented a physics-driven bowling game, worked great in the office but anything over about 8ms latency caused massive issues as the networked data was more than one frame behind.

20 Hz rate can be tolerable, 50ms round trip works if people are on good, modern connections and aren't too far away on the globe. 60 Hz is 16ms, which is often 1 graphics frame but for people with gaming monitors often 6ms is the better framerate target.

In any event, if you don't have it already, be sure you're testing under “typical" and “bad” network conditions. 200ms and 5% loss might be “typical”, and 500ms with 20% loss might be "bad" test settings, or something else that fits your game.

razcodes said:
Then why would I want to sync clocks? Why would anyone want to do it?

It depends on the type of game.

A turn based game the synchronization is your turn. It might be chess or a dungeon explorer, you can transmit the state as it exists on turn 24, the move, then the state on turn 25.

In a competitive action game or shooter, the difference between clients is far more important. Round trip time begins to matter, because what one person sees is going to be different from what another person sees, and something has to reconcile the differences.

If there is no accounting for time and you've got a star architecture, a client notifies the server then the server notifies the client. Assuming we don't have any other delays or prioritization, client A sends their position to the server with a latency delay (perhaps 75ms), the server has a tick to process it (we'll say 20Hz so it might take 16ms), and then the server sends the update to client B (perhaps another 50ms), and finally Client B processes it on the next tick which is also up to 16ms. Then you're waiting a graphics frame for it to render and present on screen, another 16ms. So 75 + 16 + 50 + 16 + 16 = 173 milliseconds. So the action shown on Client B is about 11 graphics frames behind what was done on Player A. Then the process returns as Client B's response is transmitted through the chain back to Client A, another 173 milliseconds later. Now Client A sees what happens, and from their perspective, B took advantage of where A was about a third of a second ago, which feels like ages and ages. It feels like the other player is cheating simply by being out of date through latency.

Depending on the game that might not matter. A casual turn based game it isn't a problem. Notification of events that happened like a Tetris game where you occasionally queue up dropped blocks, the drop might happen on a slightly later piece but it isn't normally critical to gameplay. On a competitive shooter, however, that same time feels like an eternity. They might be killed by a position they used to be at but no longer are; they might have seemingly dodged a headshot because they were gone on the opposite side but not the local side. Resolving that is quite complex.

If you're in a game where it matters, with competitive shooters, brawlers, MOBA, and other action combat games, highly competitive players can see an enormous impact. Network jitter can absolutely destroy top-tier players, and latency differences between players of high skill has a very strong, almost linear correlation with loss. Give them a latency difference of more than about 20ms and it becomes a clear play advantage to the lower-latency machine.

In those games, you need to do everything you can to mitigate the effects of latency between clients. That's why you see it in major engines like Unreal and Source and see it (or don't because it isn't public) in Halo and LoL in their own engines take a lot of effort to reduce the harm it can do. Even so, it is about containing the harm because it cannot be prevented entirely.

Thanks for your answer, I have tested the platform in the worst possible conditions (as a side: it is not a game, the aim of the project is to create a personal platform for developing FPS multiplayer games focused on robustness for a game engine that currently does not support it out of the box).

We are talking 250 ms (50ms jitter) + 10% packet loss. The experience is super smooth (obviously delayed) although the worse the conditions are the more bandwidth required due to redundancy data being sent to fill the gaps due to out of order/missing packets also the server and client keep a small buffer of data that adjusts depending on network condition so there is always something to show.

In the current setup (which I believe will fall under the “server ahead of the client” category), with client side and server side buffers, there was no need to sync a clock, the client sends inputs at a 60hz, the server process them and broadcasts back a snapshot every 20hz, when clients receive this data they interpolate the world state and that's pretty much it.

I think you are more worried about latency and how long it takes for an action in Client A to appear in the screen of Client B, I think that is a valid concern but the system is designed to try and reduce as much as possible the size of buffers so that the “queue” of pending packets is as small as possible, with "normal" conditions (200ms 5%) it will take around 350ms from action on Client A to Client B, keep in mind I'm trading latency for experience, yes an extra 150ms might seem much but the experience on both clients is super smooth, with better conditions (100ms of latency) it will take around 150ms from A to B.

Now the real problem (which I believe is connected to how I'm handling packets and time): player to "physics object" interaction, I have a physics object that is handled by the server and its information is contained with every snapshot broadcasted to clients, every clients sees this object in the past and can locally interact with it, but this is creating all sorts of problems with client side prediction when the user touches this object, since the object is in the past for the client, all predictions will be different from what the simulation on the server says, and I've been banging my head on the desk for months up to this point trying to come up with a solution.

If you know any books / in-depth resources (not the high leves ones like Gabriel/Valve/Overwatch) I would love to know so I can keep studying the subject till I manage to make it work.

raz0328 said:
I have a physics object that is handled by the server and its information is contained with every snapshot broadcasted to clients, every clients sees this object in the past and can locally interact with it, but this is creating all sorts of problems with client side prediction when the user touches this object, since the object is in the past for the client, all predictions will be different from what the simulation on the server says, and I've been banging my head on the desk for months up to this point trying to come up with a solution.

The short form is that you can't simulate physics across a non-LAN network and still have local interactions. The latency is too big and it will feel rough at best like it is fighting the player, and at worst behave very badly with skips and jumps everywhere.

It is possible (but not ideal) in a LAN or single-frame latency scenario where you're only very slightly out of date. It will still pop and stutter as you send your input over the network, receive last frame's position, send the motion, and then next frame get the updated position. One frame out of date feels jarring but not too bad. More than that and it really breaks down quickly.

Often the best is that once something is targeted by physics and a player for the physics to stop simulating it on the server. Basically the client becomes a streamer at that point, spewing out the updates until they're done interacting with it. But of course details will depend on the game.

There are lots of terrible scenarios for that, like multiplayer games where everybody jumps on top of physics simulated and also player controlled object such as a vehicle. Players drive the vehicle and send updates to the server, the physics of the various people riding on top of the physics object which isn't being driven by the server's physics and people suddenly experience all kinds of popping, falling off, snagging, etc.

frob said:
The short form is that you can't simulate physics across a non-LAN network and still have local interactions.

How can this be done in GMod then? When I say physics object I mean a RigidBody that players can push, stand on top / etc. I know many multiplayer games were you can get on top of a moving platform…

raz0328 said:
I know many multiplayer games were you can get on top of a moving platform…

Yes, that is true games do it. The thing gets physics interactions and players can, to varying degrees, have interactions with it. Do you also understand the engineering that goes behind it?

The short form is that you can't simulate physics across a non-LAN network and still have local interactions.

I would improve that by adding “… without some compromise somewhere.”

For example, you can optimistically allow the local physics interaction, and assume the server will do the same thing, and rewind and replay and snap to a new state if the server says something else. The compromise you accept there is “if I'm interacting based on a wrongly-predicted state, I will see snapping locally.”

Typically these systems will have a history of simulated physics states for each object, and when they receive some state or interaction “in the past” compared to their view, AND that state mis-matches what they already have, they will rewind physics to the state right before then, and re-simulate. Fancy systems keep track of which objects interact with which other objects, and only re-simulate what's actually needed. (I worked on one of those a long time ago, which scaled all the way down to a 56 kbps modem …)

You'll typically also want to classify actions into “locally simulated” versus “always remote.” For example: Turning an aircraft carrier takes a while, so you can send the request to the server, and only actually start changing the local carrier when the server tells you it's taken effect.

Anyway: Do you need to “synchronize clocks?” What actually matters, is that all systems eventually show a state that's based on all events being applied in the same order. Exactly how much divergence from this you accept, and for how long, will dictate what kinds of design approaches you choose, and there are points on this spectrum ("never any discrepancy, never any command latency") that cannot be achieved outside of a well-connected local network with zero packet loss.

In general, it's useful to agree on the sequence numbers of simulation ticks. When the client says “I pressed the brake on my tick 1234” then that should be meaningful to the server, and when the server then says “the ball bounced in front of the car at tick 1233 because it hit another moving player at tick 1229" that should be meaningful to the client (for re-simulation.) The relationship between wallclock time and tick number then needs to be established and, at times, adjusted (because clocks drift) and you could count this as “synchronizing clocks” if you want.