While having simulations match each other precisely is an admirable goal for some games, it comes with a steep computational and performance cost.

Every game I've worked on, and nearly every fast-paced game I've read about, follow a model where everybody does their own thing and continuously corrects to whatever source they consider authoritative.  Close enough is usually adequate, for the few things that require round-trip verification you can often mask the latency with a mix of audio and animations.

There is significant difficulty even in making the CPU instructions work deterministically. SSE floating point operations have been a massive improvement in this area between machines, but still many math functions like trig operations give various results both between machines and even in different locations in the code. The optimizer is also not friendly for keeping other operations deterministic and properly sequenced everywhere.  It can be done, but I don't envy anyone who has to do it.

I agree that full determinism is hard, and only worth it sometimes!

However, many physics engines use low-quality integrators, which are time step dependent. Then you get situations like in the original Quake series where some ledges were reachable by jumping if your computer ran at a fast enough frame rate, but not if it ran at a slower frame rate.

Additionally, in spring/dampened contact systems, the depth of penetration for a "resting" object depends on the time step size. If the time step size suddenly changes, the object will "bounce" and suddenly get ejected out of whatever it is resting on, because the equilibrium between gravity acceleration and contact expulsion is disturbed.

Thus, fixing the time step size for the simulation (and running multiple timesteps between render frames if needed) is an important tool to use in making the simulation "fair," and is something I wouldn't really want to discard. Even if the simulation is nondeterministic (in the bit-pattern floating-point values sense,) a fixed size time step leads to both better stability and more fairness in movement.

On 2/6/2019 at 3:17 AM, Hodgman said:

We do a clock-synchronization process during connection to get a common wall clock for each client and the server to refer to, and then the clients can adjust their in-game clock appropriately to maintain themselves in the future slightly.

I've been trying to understand how jitter is handled myself after reading Gaffer's article on state sync. He explains the reasons why he delays but nothing specifically about how the client chooses it's time base. Could you elaborate on how you do this a little? eg what amount of time you spending choosing the right number and how you handle resolving if the time drifts and the client is ahead/behind?

The server can send information in each response packet. Client will include target time for each packet, and server simply responds with how early / late the packet arrived. The client can then adjust as necessary, say aiming for a 25 MS early arrival on average.