Heyo, I'm making a custom engine that uses 2:1 isometric sprite graphics with a 3D world simulation. Each render tick, I convert each entity's world position into a screen position, and render their sprite appropriately:

While moving diagonally across the screen, the naive rendering approach (using the raw world -> screen pixel coordinates) caused some jitter. I resolved this by making sure that during diagonal movement, the sprite always gets rendered on the same pixel in the 2:1 pattern. For example, below is a diagonal line with a repeated 2:1 pattern. The first pixel in each step of the pattern is colored blue. If the sprite happened to start its diagonal movement on the first pixel in the pattern, I would continue to round its origin each frame to the closest blue pixel.

This gives me a sprite that looks like its traveling in a straight line, with no up/down jitter. It introduces a new issue though: if the entity's runspeed doesn't happen to perfectly line up with the pattern, it will sometimes move by 1 step, and sometimes move by 2:

This gives a slight jitter to the graphic, as it moves irregularly across the screen. This jitter isn't present if I choose a runspeed that moves the graphic a consistent number of steps each frame.

My question is: is there any fix for this? Perhaps a different movement algorithm that'll select pixels more optimally? Or a different approach altogether?

Net_ said:
My question is: is there any fix for this? Perhaps a different movement algorithm that'll select pixels more optimally? Or a different approach altogether?

The only way to have constant velocity movement without any jitter requires a lot of constraints, which dominated in the 8/16 bit era:
Rendering must match constant display refresh rate.
Sprites can only move 1 px each Nth frame, or N px each frame. So there is only a limited set of different speeds you can use.
Same for the direction of motion. If it's 2D, the above applies to both dimensions.

If some of your sprites have acceleration, meaning velocity isn't constant but changes gradually, rounding jitter is unavoidable if the rendering is quantized to whole pixels. (But the rounding also becomes less noticeable with variable speed.)

Technically, the solution is subpixel accuracy for rendering, so no more snapping to quantized pixels is needed and all constraints go away.
That's easy to do. Just allow your sprites to render at real coordinates instead integer coordinates, and use texture filtering.
Then the movement at any constant speed should look smooth, but at the price of varying sharpness of the sprites, as depending on subpixel position more or less mixing of texels is needed for the filter. But with your artstyle that's probably not visible, so i would try it.

For fast moving sprites, motion blur would be an option and it's cheap for 2D. I wonder why seemingly nobody does it.

look and search : bresenham algorithm

and that https://www.geeksforgeeks.org/bresenhams-line-generation-algorithm/

JoeJ said:

Technically, the solution is subpixel accuracy for rendering, so no more snapping to quantized pixels is needed and all constraints go away.
That's easy to do. Just allow your sprites to render at real coordinates instead integer coordinates, and use texture filtering.
Then the movement at any constant speed should look smooth, but at the price of varying sharpness of the sprites, as depending on subpixel position more or less mixing of texels is needed for the filter. But with your artstyle that's probably not visible, so i would try it.

Thanks a ton for mentioning this, I was originally rendering at subpixel coordinates and was still getting jitter, so I figured I needed to find my own solution. It turns out, I wasn't properly setting the filtering to linear! (More specifically, I use SDL2 and didn't realize the hint needed to be set before creating the textures).

It definitely adds some blur, but it may not be noticeable once I have a real scene set up. I'll go with this for now and revisit later. Here's a comparison of linear vs nearest:

Net_ said:
It definitely adds some blur, but it may not be noticeable once I have a real scene set up.

In theory it should work to use a sharpening filter as post process, but in practice this also introduces new problems with high frequency content like foliage.
You could try it with Reshade, Fidelity FX, etc.

Alright, subpixel sprite positions + texture filtering got rid of the jitter! The texture filtering introduced two more issues that I fixed:

1. Vertical black lines related to the tile sprites
   1. Fixed by adding 2 pixels of padding between sprites in the sprite sheet.
2. Grey lines around all the edges of the floor tiles, where the colored pixels met transparent pixels (alpha cutouts)
   1. Fixed by switching to a premultiplied alpha blend mode function for the tile sprites (the textures already happened to be premultiplied).
      1. Excellent reference: https://stackoverflow.com/a/71825982/4258629
      2. Another: https://shawnhargreaves.com/blog/premultiplied-alpha.html

I have one last issue remaining. On the corners of some of my sprites, where one color sharply turns into another, I get a single column of differently-colored pixels when the sprite is at a subpixel coordinate. You can see the faint line between the sides of the boxes in this image:

Here's a closeup of where the two colors meet at the corner of the box. You can see the filtered color that's causing the line:

The vertical line isn't too noticeable in still images, but it jitters as you move around, making it unpleasant to look at. Is there a way to solve this? Or do I have to draw my sprites such that there's no sharp changes in color where the sprites will join?

Net_ said:
I get a single column of differently-colored pixels when the sprite is at a subpixel coordinate.

You need to dilate the colors over the border. So you need one more row / column of duplicated / reflected pixels on each tile.
For example, 8x8 tiles become 10x10 tiles:

You still use only the inner 8x8 section of the image, but the additional border of 1 pixel on each side guarantees the filter knows the right color across the border.

This is assuming you use an texture atlas, so multiple tiles go into one image.

If you use a unique texture for each tile, you could use the proper texture warping mode, which is GL_REPEAT for my example.
See https://learnopengl.com/Getting-started/Textures

Unfortunatedly it is not so easy for isometric tiles, so this might cause you some work on assets or an automated tool to add the borders automatically. But the idea remains the same: If you know the tile is likely to repeat, you pick the pixel from the opposing edge of the same tile. If you don't know the adjacent tile, just duplicate the color on the current edge.

But most important: You can select the method of filtering individually per texture. So you could use bilinear filter only for the sprites, but unfiltered nearest for the tiles. So they would work as before.
However, if yopur game should work at different resolutions requiring some scaling, the filter might be needed again and so the work on the borders.

JoeJ said:

You need to dilate the colors over the border. So you need one more row / column of duplicated / reflected pixels on each tile.
For example, 8x8 tiles become 10x10 tiles:

To clarify, this is on the inner part of the sprite, not on the borders of the image. Here's the box sprite in question, the problematic part being the vertical line in the middle that divides the two sides:

Net_ said:
To clarify, this is on the inner part of the sprite, not on the borders of the image. Here's the box sprite in question, the problematic part being the vertical line in the middle that divides the two sides:

Hmm… no, i think it is about the border. If we compose two cubes:

The border of the orange tile draws over the blue one. So it's likely the orange border is the culprit?

Edit: Btw, i just see your alpha is probably a bit less then one for the ghost sprite, explaining why it's darker than the unfiltered version. And i can see the background slightly shining through.

JoeJ said:

The border of the orange tile draws over the blue one. So it's likely the orange border is the culprit?

Ah, I see. I tried continuing the color for 2 pixels on each side and unfortunately it didn't affect the line artifact.

Something I just noticed is that the filtered color line gets rendered to the left of the corner:

So even if the next sprite was to perfectly overlap, this filtered color would still be visible. I'm not sure what to do with that information, though. I can move the line 1 pixel to the right and that gets rid of the artifact in that direction, but makes it more visible when the boxes are overlapped on the other side:

The line no longer jitters though, so this may be an acceptable solution. For actual wall sprites, they're only going to be tiled in one direction so this would work fine.

Good point about the ghost sprite, that makes me feel better!