OK so just to update on my current situation, to solve the problem of a new message coming before the interpolation finished and to fix the jittering what I did was to create a queue, now that obviously creates issues the more messages arrive the longer the queue and the longer the delay, also dependent on how much of a delay one adds. I am creating a pong game so one could say that it's a reflex game and @Kylotan in his original post mentions that this technique wouldn't work on those. 

Is there a way to do snapshot interpolation for reflex games?

You might want to look at the Entity Position Interpolation Code library:

https://github.com/jwatte/EPIC

We used a type of extrapolation for a game we created. It works well on slower games and masks latency effectively: http://www.kinematicsoup.com/news/2017/5/30/multiplayerprediction

Thank you both! I added both urls to my read list and already started going through the first. I'm not sure extrapolation is the right way for a pong game because of the skill involved if it gets corrected it might look like the other player isn't making up his mind. 

It's not clear why you didn't try the simplest solution to the original problem: when a new future position comes in from the server, you lerp directly towards that from wherever the client representation is now. In other words, the currently lerped position becomes the 'now, T=0' position and the received position becomes the 'later, T=N' position, where N is some number of milliseconds.

The reasoning for using a time in the future is because of the following:

• if you instantly snap to whatever position the server reports then it's obviously going to look very jerky - so you want to smooth that over some period of time. (Let's call that a 'time buffer' for the purposes of this post.) You interpolate between a start position and a next position.
• that time buffer effectively adds more latency, because your client has that extra delay before it moves the entity where it needs to be. So you want the buffer to be as short as possible to reduce this.
• On the other hand, you always want to have a 'next position' queued up in the future or the entity will stop moving while it waits for the next snapshot to come in. (If you think about it, the 'instant snap' situation can be considered a special case where the interpolation is over 0 milliseconds.)
• So, you choose a time buffer length that is short enough to not add too much extra latency, but long enough that you always have a next position from the server queued up, even in the face of varying message ping/latency (known as 'jitter')

Regarding Pong - I think this method would be fine. You have very few entities to update and therefore very low bandwidth requirements, so I'd just suggest sending messages as frequently as possible.

@Kylotan I did try to "override" the current position but still was getting jittering maybe if I had stopped and optimized how much into the future it should jump rather than changing to a queuing system would hide the jittering. I might give it another try to see if I did a mistake at the time. 

as you say the extra latency started creating issues even if it was set to 50ms. I currently solved that by speeding up the lerp but that also doesn't create a very smooth result but a passable one. I realize that I l get better results the more I play with the numbers but shouldn't there be a result that makes it look like it's running 30 fps? Currently I'm sending a packet every 2 frames.

If you were seeing discontinuities in the position, then you weren't treating the currently interpolated position as the new start position, or you weren't resetting the interpolated time at that point.

If you were seeing the object continually stop and start, then your updates were arriving too late to ensure that there was always a future position queued up.

One good way to debug this sort of issue is to slow everything down - drop the network send rate to something like once per second, set your update rate at something like 20 or 30 frames per second, scale up all other time values accordingly, and do a lot of logging to see when things change in ways you don't expect. Get it working correctly at the slower speeds, where it's easier to debug any issues. Then ramp up the speed to what you actually need.

Also, ensure you understand how to interpolate properly. If your interpolation looks like Lerp(start, end, 0.5) (or any other constant) then that is not truly interpolating linearly between the values over time.

If this isn't enough information to help you, then you will probably have to show your snapshot handling and interpolation code.

It might've been a case of a bug in the server code as well which didn't send packets in the same order they were sent. I added an indexing method when I did the queuing to drop "previous" packets. 

OK so my jittering in the end wasn't due to mistakes in snapshot interpolation code rather some extra physics were interacting with my object when  they shouldn't thus network code was setting the object where it should be and the physics engine tried to move it elsewhere, it could also be what I mentioned above about indexing my packets but mostly the physics caused the issue. 

To comment on @Kylotan 's previous messages, the reason I asked why we add the future was because it felt unatural to manually add extra latency but of course there isn't any other way to interpolate. I think what would make your answer more complete would be to increase the speed of interpolation by something small so that it eventually catches up to the extra latency that was setup, basically that was what confused me as I thought for skill games 50ms might create edge cases but when I fixed the physics issue I saw that it won't.