Hi all,

I've built a very basic engine & game (not really a game yet) in Java to learn OpenGL. I didn't use C++ as I wanted to learn one thing at a time & I'm already very comfortable with Java.

I've used LWJGL just as an OpenGL wrapper, I've not used anything else it offers - I wanted to learn & write everything from first principals.

Anyway, it's a very simple game loop that renders a single square which bounces around the screen linearly. Clicking in the window will create another square with a random size/colour/velocity at the cursor location.

Now to my question, I observed that every now and again the square has some serious jitter for a few seconds - looks like frames skipping or something.

Also, generally, the square does not seem to move very smoothly, even though it's running at 60 FPS - increasing this does not seem to improve things.

What I've found;

1. Whether I have one square or 200 squares the performance remains constant and the problems still exist. The issue is as obvious with one as it is with many.
2. There are no obvious memory leaks or CPU issues - I attached a profiler, everything looks OK.
3. It's not GC - again, I can see this in the profiler. I experience jitter GC is not occurring, so I can rule that out too.

I recorded a video to demonstrate the issue (it can be clearly seen at 16 seconds): https://github.com/arrwhidev/opengl-game/blob/master/jitter.mov

The code is on GitHub and pretty easy to run, details in README: https://github.com/arrwhidev/opengl-game

Any ideas where to start? Any clues/ideas?

Many thanks!

I just looked at the video. It's like if your computer was doing something intensive when this jitter occurs.

You can try to log your timer and check if it's quite regular or if you have a bigger time lap when this happens.

Do you use double buffering ? How do you draw (old school glBegin or display lists, draw arrays / elements with VAO) ?

When you have many quads, do the jitters happen at the same time, or each quad will have its jitter separately ?

This may be too obvious a question, but do you have v-sync enabled? (The behavior you're seeing may be unrelated to that, but it might still be useful to know.)

I had a look through your code, and I think I have an idea why this jitter happens.
From what I could gather, you appear to be using a fixed logic game loop running at 60hz, with the render rate capped by v-sync most likely also 60hz in your case.

So here is my two cents, In the game loop it accumulate the frame time once every frame, this is then checked and subtracted in the while loop that runs your logic update. However it subtracts the "expected frame time" from the accumulated time, while this improves the timing a bit it also tend to accumulate a little bit of leftover time every frame, then after a short while it will have accumulated enough time to run the logic update twice for a single frame, thus skipping the next hence the frame skipping you experience.

A bit more in detail:

accumulator += frameTime;

// Logic update
while (accumulator >= DT)  {
	accumulator -= DT;
	engine.update(DT);
}

After First frame:
accumulator = frameTime - DT

After Second frame:
accumulator = (frameTime + leftoverTimeFromLastFrame) - DT

After Third frame:
accumulator = (frameTime + leftoverTimeFromTheLastTwoFrames) - DT

Many frames later:
Now the accumulator has accumulated enough leftover time that will now be greater than DT, even before adding frameTime. And here is the issue, because accumulator now hold more than twice the value of DT, the while loop runs twice, processing two frames at once and because rendering runs at the same rate as the logic, it will miss that frame.

This isn't necessarily a bug though, that's how it's supposed to work, so my advice is that you keep the logic rate lower than the render rate, possibly around 30hz, then interpolate between the last and next logic frame, on every render.

Other than the above, I noticed you are using "System.currentTimeMillis()" for the game loop, you might be better off using "System.nanoTime()" or the one Lwjgl itself supplies "GLFW.glfwGetTime()" since these are likely to have a better timing accuracy, something that also might contribute to the jitter.

This guy above ^ is 100% right, include the time between frames, you can also take an average of last frame and now time diffeeence divided by 2

22 hours ago, _Silence_ said:

I just looked at the video. It's like if your computer was doing something intensive when this jitter occurs.

You can try to log your timer and check if it's quite regular or if you have a bigger time lap when this happens.

Do you use double buffering ? How do you draw (old school glBegin or display lists, draw arrays / elements with VAO) ?

When you have many quads, do the jitters happen at the same time, or each quad will have its jitter separately ?

I'm using VAOs/VBOs to render. When multiple quads are visible, they all jitter at the same time, so it does seem like it's related to the game loop, as  @magn919 wrote.

22 hours ago, Zakwayda said:

This may be too obvious a question, but do you have v-sync enabled? (The behavior you're seeing may be unrelated to that, but it might still be useful to know.) 

V-sync is enabled, I tried turning it off but it didn't appear to improve the overall smoothness and jitters still happened too.

19 hours ago, magn919 said:

So here is my two cents, In the game loop it accumulate the frame time once every frame, this is then checked and subtracted in the while loop that runs your logic update. However it subtracts the "expected frame time" from the accumulated time, while this improves the timing a bit it also tend to accumulate a little bit of leftover time every frame, then after a short while it will have accumulated enough time to run the logic update twice for a single frame, thus skipping the next hence the frame skipping you experience. 

Thanks for your detailed response @magn919. What you've said makes a lot of sense and I agree that this is probably the cause.

I re-wrote my loop to use a fixed update rate with variable frame rate, setting the update to 30 caused what I would expect; the movement became very choppy. The jumps between updates were clear. I upped the rate to 300 and became smoother.

Then I implemented a very basic interpolation, by guessing the position between old and new position using the interpolation value (between 0 and 1). It did smooth things out - still isn't 100% perfect though - still seeing frame skips sometimes - it seems less frequent though.

I've been staring at this bouncing square for so long now that I don't even know what looks good anymore haha!

Also, my simple interpolation isn't very smart, it only works for position, and linearly. If entities were to accelerate, this code would predict too much or too little. Also, if entities were not moving, but scaling, or rotating, it would do nothing. This means I need to duplicate all my coding efforts now; as I add things into the game logic (such as scaling and rotation), I need to remember to replicate these in my interpolation code.

Is this correct? It feels wrong and a bit smelly. However, I can't think of any alternatives.

BTW - I pushed changes to a new branch; https://github.com/arrwhidev/opengl-game/tree/fix_gameloop

Thanks again.

Final update before bed, I realized vsync was ALWAYS on. I've turned it off now.

Even though I'm updating at 30hz, rendering at something like 400hz and interpolating, it's now worse that it ever was, jitter and frame skipping all over the place.

All this is visible in the fix_gameloop branch.

Dammit!

Hmm, you shouldn't really be calling System.currentTimeMillis() more than once per frame, then reusing the result.
Try reverting to the game loop you had before, but keep the update rate at 30hz, then calculate the render interpolation like this.

engine.render( accumulator / DT );

Then if you aren't already doing this, on your entities you keep a copy of some state from the last frame, like position, then linearly interpolate between last and current frame using the 'interpolation' variable, here is a little example.

public void render(double interpolation){
  Vector2f entityLastPosition = someEntity.lastPosition;
  Vector2f entityCurrentPosition = someEntity.position;
  Vector2f renderPosition = Vector2f(entityLastPosition).lerp(entityCurrentPosition, interpolation)
  renderEntity(someEntity, renderPosition);
}

42 minutes ago, arrwhidev said:

frame skipping all over the place

In the light of what other discovered in your code, it's unlikely that you have frame skipping.

You should have a search on these forums about "game timer loop" or such. This had been covered in a well manner. Alternatively, google raised me, what it seems, good pages, in the top of its results: this one for example.

On 7/24/2019 at 12:28 AM, magn919 said:

Hmm, you shouldn't really be calling System.currentTimeMillis() more than once per frame, then reusing the result.
Try reverting to the game loop you had before, but keep the update rate at 30hz, then calculate the render interpolation like this.

engine.render( accumulator / DT );

Then if you aren't already doing this, on your entities you keep a copy of some state from the last frame, like position, then linearly interpolate between last and current frame using the 'interpolation' variable, here is a little example.

public void render(double interpolation){
  Vector2f entityLastPosition = someEntity.lastPosition;
  Vector2f entityCurrentPosition = someEntity.position;
  Vector2f renderPosition = Vector2f(entityLastPosition).lerp(entityCurrentPosition, interpolation)
  renderEntity(someEntity, renderPosition);
}

 

I've tweaked a bit as you say but I'm still in the same position. It's kinda jittery entirely smooth. Everything is pushed to branch, but I'll post here for ease.

Game loop: (https://github.com/arrwhidev/opengl-game/blob/fix_gameloop/src/main/java/com/arrwhidev/opengl/engine/loop/GameLoop.java)

private static final int UPDATES_PER_SECOND = 30;
private static final double NS_PER_SECOND = 1000000000.0;
private static final float NS_BETWEEN_TICKS = (float) NS_PER_SECOND / UPDATES_PER_SECOND;
private static final float DT = 1.0f / UPDATES_PER_SECOND;

private long now() {
  return System.nanoTime();
}

public void run(Window window) {
  double nextUpdate = now();
  float interpolation;
  while (isRunning && !window.windowShouldClose()) {
    long t = now();

    while(t >= nextUpdate) {
      engine.update(DT);
      nextUpdate += NS_BETWEEN_TICKS;
      updates++;
    }

    interpolation = (float) (t + NS_BETWEEN_TICKS - nextUpdate) / NS_BETWEEN_TICKS;
    if (interpolation > 1) interpolation = 1;
    else if (interpolation < 0) interpolation = 0;

    engine.render(interpolation);
  }
}

Interpolation: (https://github.com/arrwhidev/opengl-game/blob/fix_gameloop/src/main/java/com/arrwhidev/opengl/game/ecs/misc/Transformation.java)

float newX = ((position.x() - position.prevX()) * interp) + position.x();
float newY = ((position.y() - position.prevY()) * interp) + position.y();

return MODEL_VIEW_MATRIX.identity()
  .mul(camera.getProjection())
  .translate(newX, newY, 0)
  .scale(position.getScale());

On my machine that runs at 30 update/sec and around 700 frames/sec.

I've uploaded another video to show how it looks for me. I feel like I've got everything right. But it's definitely not smooth: https://raw.githubusercontent.com/arrwhidev/opengl-game/fix_gameloop/jitter2.mov

Thanks again!