Hello guys,

in the past few weeks I read a ton of material about networking a multiplayer game, prediction, extrapolation,interpolation and whatever. I tried to implement my own solution to predict the movement of the player-controlled character and make sure it is validated by the server, but I have some jumpy movements that, as far as I understood, shouldn't be there.

What I'm doing right now is pretty simple: every client tick I record the state of the input, assign and incremental ID, and send it to the server. At the same time I simulate the client. When the server receives the command, it simulates it and send back to the client the new position together with the id of the just simulated command. Once the client receives the ACK packet, I assign the server position to the player, and then apply all the commands recorded, starting from the last acknowledged one. World state sync is done with a separate message at lower rate, and does not include the player position.

Simulation runs on both client at server at fixed 30fps for now.

When simulating variable (but still acceptable) latency, the player jumps around a little bit. Is it due to the fact that I should take into account the latency somehow when re-simulating the client commands when I receive the ACK packet?

I thought about two alternatives to solve the issue.

The first one is to set an acceptable distance between the client-side position and the acknowledged one: if we are in this range I keep the client where it is. The server will still be autoritative and all the collision and such will happen correctly, and the client won't notice any jumping around.

Or I could periodically send just the client position to the server, and validate the new position there using simple sweeping to check for collisions and such. As long as the movement is linear from point A to B, it should work fine?

Btw, the prototype I'm working on is something similar to realm of the mad god but with less bullets, just to give you and idea about what I am aiming for ;)

Thanks.

Do you smoothly lerp to the corrected position? Does the server know that the input arrived was actually generated at receiveTime-ping/2? Does the client rollback time when receiving correct positions to the time the server said it generated those? Do you drop corrections that arrive after a newer correction has been received (out-of-order packets)? Have you checked if the server receives all of the clients inputs, and if not informs the client of what inputs got accepted? Sending input to server should work fine, and as long as both client and server know everything that might affect position, there shouldn't be any difference as long as neither input nor correction packets get dropped. If both systems can simulate the outcome of the users input the same way, there should be zero difference for a bit of time - until indeterminism/floating-point-inaccuracy kick in.

Do you smoothly lerp to the corrected position?

No, at the moment I'm setting the corrected position directly. Problem is that the corrected position seems to be behind so I always jump back. That's why I thought it might be something related to not taking into account the latency. If is smoothly lerp to the correct computed position, and the position is behind, want the client slow movement down while applying the correction?

Does the server know that the input arrived was actually generated at receiveTime-ping/2?

Yes, although I execute the command on the current position and I don't roll back in time the command queue on the server. Should I do that too? Or shouldn't I simply consider the roundtrip time while re-apply the commands on the client?

Does the client rollback time when receiving correct positions to the time the server said it generated those?

Yes. I mean, I set the client position to the one acknowledged by the server, and then re-apply the commands starting from the one the server just sent me the ACK for.

Do you drop corrections that arrive after a newer correction has been received (out-of-order packets)?

Yes.

Have you checked if the server receives all of the clients inputs, and if not informs the client of what inputs got accepted?

Actually, I'm sending them via a unreliable but sequenced channel. Maybe I should continuously send all the commands queued up until the one I received the acknowledgement for?

until indeterminism/floating-point-inaccuracy kick in

Yeah, that's something I will try to deal with at second time (or I will simply cross fingers and hope it won't turn out to be a huge deal), but as far as I understood if I do the command queue approach properly it should not be needed (assuming everything else is properly syncronized)

There are a few things that you must be careful of. My game movement code, for the record, looks like this:

https://github.com/agoose77/PyAuthServer/blob/master/game_system/controllers.py (PlayerPawnController)

The client creates a move:

• Applies inputs
• Saves inputs,
• Processes physics
• Saves physics state
• Creates a struct of the physics state and inputs state (so that the physics state is the results of those inputs)

And sends it to the server. The server buffers this move (a queue, basically).

In the update tick, the server pops a move from the queue, and processes it. The queue is artificially delayed so that we have at least 100ms (or arbitrary time) of input data incase of jitter (variable latency).

This problem (which you may have overlooked) means that the server won't receive moves regularly, which means it will think the client mis-simulated certain moves (if you send the correct position from the client, but the server disagrees because It didn't simulate a movement for a specific tick).

The server then uses this move to process client inputs and compare the results of the simulation to the client. If the client didn't produce similar results (of both rotation and location) (not dynamics, as errors will be seen in the location/rotation later on), we send a correction to the client. The client will be X ticks ahead now (RTT + server buffer latency) and so will lookup the old move, and resimulate the inputs from this move.

At this point you must also consider the fact that once you make a correction for move T, there will most likely be corrections for another [RTT / 2 + server buffer latency] ticks, because it still receives old moves "in the wire" and in the buffer. So, you should ignore any corrections until this time has elapsed. The safest way to do this is have the client save the current outgoing move ID when it makes a correction, so that later corrections from the server are rejected if their ID is less than the ID which includes the correction. Alternatively, (as seen in my code, though I'll change this), just have the server send one correction (the full state), and wait for the client to tell it that it has received that correction. Due to the possibility of variable packed order, this is less of a good solution.

You should also handle what happens if a move isn't received between received moves (T arrives, T + 1 arrives, T + 2 doesn't arrive, T+3 arrives ...). Because of the RTT, re-sending doesn't make sense; it won't arrive in time, so often the best solution is sending moves redundantly.

If is smoothly lerp to the correct computed position, and the position is behind, want the client slow movement down while applying the correction?

Well yes, but it would still be correct. However, first try to minimize errors, and then smoothly apply the corrections you cannot get around. If the server says the position is back there, then it is back there - if that means client seems to slow down, then that's how it is, better than having the position suddenly jump back.

Should I do that too? Or shouldn't I simply consider the roundtrip time while re-apply the commands on the client?

Send client input with a timestamp a bit in the future like 50ms (depends on rtt), remember to also apply that to client - meaning store inputs and apply them a little later. That or rollback on the server. If you just consider roundtrip when applying again on the client ofc there will be differences, because for the client you have applied the same input in different times/frames = different resulting positions.

Maybe I should continuously send all the commands queued up until the one I received the acknowledgement for?

You just send em out as they come, don't wait for acks. maybe send important inputs (press fire-button or smth) as reliable, but i'd say that's it.

What you should do to track down the source of the jitter is print out exact player input on client and server with current gameTime and positions. then compare those tables and check if same input at the same gametime results in same positions on client and server, and if not what is the difference in time. that should tell you if you are off-by-rtt somewhere.

EDIT:

@Angus i think the client should check for corrections, not the server. the client just sends it's inputs, the server sends results, the client compares to own results. client's upstream is most expensive bandwidth, server's downstream cheapest, and in a world with lets say 1000 objects that need possible corrections you wouldn't want clients to have to send possible "moves" for all of them, nor the server calculate corrections for all of them. apart from this system being somewhat exploitable by manipulated clients - pure input is much easier to sanitize then possible moves.

There's another method for keeping the server authoritative while respecting client movements so that you don't have be corrected all the time if your simulation is deterministic.

For every tick:

• save the simulation state all the players in a cyclic buffer. (both server and client)
• handle input on client, send the input and tick number to the server (even better, 'send' a packet to yourself so you only have 1 place where you do movement code which applies to all players, make sure you get the packet before you start simulating for that tick)

For the first input message you receive, you record the local tick count and tick count that's in the message. Apply the input to the simulation.

For all other input messages you receive, you use the previously recorded tick counts to determine whether it happened before your current tick count or after. If it's before, use your state buffer to 'rewind' and then simulate forward again for the difference in tick counts. If it's after, store the input and wait with applying it for when you hit that tick count.

You'll only want to keep a small state buffer, because you don't want to rewind and resimulate that much. If a client fell back more than that, send it a force sync packet the server tells the client it's position and goes to a state where the client has to send it's first input message again. On the client side, if you got a packet that is beyond rewinding, request a sync from the server for that player.

All this requires your packets to be ordered and reliable. If you receive a packet that is out of order, don't discard it, but use a window where you store an x amount of packets that come after the first sequence number you expect and slide this window along.

Doing this will result in the client being able to move around freely with no jitters at all, however, the rest of the clients jitter a bit because you'll have to occasionally rewind and re-simulate them, but this can be smoothed out with interpolation. IMO this is preferable, because as a player you will feel in direct control without being corrected all the time.

It would be interesting to know, for diagnostic purposes, whether the cause of the jumpiness is that the server comes to a different simulation result than the client, or whether the cause is someting in your forwawrd playback.

Do you save the simulated state for time period X, and compare to what the server sends you back?

Thanks for the replies guys. I will try to apply the suggestions tomorrow and also collect some of the data and let you know if I can get to more acceptable results!

Here we go. I didn't have too much time to delve deeply into the problem yet, but I tried logging the messages and changing the network error simulation setting of my transport layer. It seems that the issue is related to the fact that some messages don't arrive from the client to the server, due to the fact that I'm using a reliable channel but I'm discarding out of order messages. As long as, at the moment, my client sends every frame a message with the input state, skipping one or more of them creates the issues (it seems).

My plan now is to make sure the packets arrive and are executed in order, and then follow the approach I was taking before to correct the client, applying smoothing and taking into consideration the latency properly.

To execute the input on the server at the correct times, I guess the only real solution is to advance the simulation on the server as normal and buffer the incoming input until I get the expected one, and then roll the player entity back in time and apply the input? Or are there any smarter ways to solve this issue?

I've a side (but I think related) question about simulation update rate: at the moment I'm simulating the physics on the server and the client at the same tick rate. This should allow me to have pretty similar behaviour (ignoring floating point issues for now as long as it is not something I want to deal with yet) assuming all the input arrives to the server an gets processed properly. What if I want to change the tickrate of the server to run it less frequently? Should I apply the input multiple times per update on the server based on the timestamp of the input coming form the client?

Thanks,

Gabriele

You generally don't want to use a different tick rate between server and client.
You generally want a single, fixed, tick rate for your simulation.

You *can* build various architectures where the server might simulate, say, at 10 Hz, and the client simulates at the same rate as display frame rate, but then you will forever fight problems where simulations are stable on one side but not the other.
Depending on how much physics you use, this may or may not be manageable.

As an example: How high you jump when given an impulse of a particular size varies by the size of the physics step rate, if you use a typical first-order gravity simulation.