I'm working through implementing TAA in my renderer and I am wondering how I might remove the jitter before I output the velocity to my texture to store. I've shown below how I am adding jitter the projection matrix and also the current calculation for my velocity values.

Would appreciate any insight into how I might remove the jitter. Thanks.

Applying jitter:

static const float Halton23[8][2] = {
    {0.0f / 8.0f, 0.0f / 9.0f}, {4.0f / 8.0f, 3.0f / 9.0f},
    {2.0f / 8.0f, 6.0f / 9.0f}, {6.0f / 8.0f, 1.0f / 9.0f},
    {1.0f / 8.0f, 4.0f / 9.0f}, {5.0f / 8.0f, 7.0f / 9.0f},
    {3.0f / 8.0f, 2.0f / 9.0f}, {7.0f / 8.0f, 5.0f / 9.0f}
};

float haltonX = Halton23[FrameIndex][0];
float haltonY = Halton23[FrameIndex][1];

float2 dimensions = float2(1920, 1080);
float2 invDimensions = float2(1.0f / dimensions.x, 1.0f / dimensions.y);

float2 offset = float2(haltonX, haltonY);

offset.x = ((offset.x - 0.5f) / dimensions.x) * 2.0f;
offset.y = ((offset.y - 0.5f) / dimensions.y) * 2.0f;

float4x4 translationMatrix = float4x4(
    1, 0, 0, 0,
    0, 1, 0, 0,
    0, 0, 1, 0,
    0, 0, 0, 1
    );
translationMatrix[3][0] = offset.x;
translationMatrix[3][1] = offset.y;


float4x4 newProjection = mul(translationMatrix, P);

result.position = mul(float4(position, 1), WorldMatrix);
result.position = mul(result.position, V);
result.position = mul(result.position, newProjection);

Calculating velocity (need to remove jitter here)

void CSMain(uint3 DTid : SV_DispatchThreadID)
{
	float2 texelSize = float2(1.0f / 1920, 1.0f / 1080);
	float2 texCoords = texelSize * (DTid.xy + 0.5);

	// Calculate current position world space
	float depth = other.SampleLevel(sPointClamp, texCoords, 0).x;
	float ndcX = texCoords.x * 2.0 - 1.0;
	float ndcY = 1.0f - texCoords.y * 2.0f;
	float4 viewpos = mul(float4(ndcX, ndcY, depth, 1.0), InverseProjectionMatrix);
	viewpos = viewpos / viewpos.w;
	float4 worldspace = mul(float4(viewpos.xyz, 1.0), InverseViewMatrix);

	// Reproject using previous view and proj matrix
	float4x4 prevViewProj = mul(prevView, prevProj);
	float4 previousPosNDC = mul(float4(worldspace.xyz, 1.0), prevView);
	previousPosNDC = mul(float4(previousPosNDC.xyz, 1.0), prevProj);
	previousPosNDC.xy /= previousPosNDC.w;
	float2 prevPositionSS = (previousPosNDC.xy * float2(0.5, -0.5) + float2(0.5, 0.5));

	// Calculate velocity: current position in SS minus previous position SS
	float4 ssPos = float4(texCoords.x, texCoords.y, depth, 1.0);
	float2 velocity = ssPos.xy - prevPositionSS; 

	// Remove jitter before output

	

	dst[DTid.xy] = float4(velocity, 0.0, 1.0);
}

Wouldn't you just subtract the current offset float2 from ssPos, and subtract the previous frame's offset from prevPositionSS before calculating velocity? (i.e. remove any jitter from both positions before calculating velocity)

That's exactly what this tutorial says to do.

@Aressera Thanks for the reply! Doing something like this, is this what you were referring to? The resulting output to the texture with the velocity value is black, even when I move the camera around. Not sure if this is how the output for the velocity texture should be since in this article they showcase the velocity texture with camera movement and it's also black, though I have seen some examples that are not black.

float4 ssPos = float4(texCoords.x, texCoords.y, depth, 1.0);

// Re-calculate the applied jitter to remove it (improve later)
float2 dimensions = float2(1920, 1080);
float2 invDimensions = float2(1.0f / dimensions.x, 1.0f / dimensions.y);

float haltonX = Halton23[FrameIndex][0];
float haltonY = Halton23[FrameIndex][1];

float2 offset = float2(haltonX, haltonY);

offset.x = ((offset.x - 0.5f) / dimensions.x) * 2.0f;
offset.y = ((offset.y - 0.5f) / dimensions.y) * 2.0f;

ssPos.xy -= offset;
prevPositionSS -= offset;

float2 velocity = ssPos.xy - prevPositionSS; 

You need to subtract the jitter from the previous frame from the previous screen space position, and current jitter from the current screen space position. Right now you subtract the same offset from current and previous frame positions, which has no effect on velocity (cancels out).

@Aressera Oh I see! I think I won't be able to do this in the way I have currently setup. Right now I am calculating the Halton offset within the shader and not passing it in as a constant buffer/uniform. I think I would need to pass the currentOffset and previousOffset after calculating it onto the CPU and pass it the GPU?

I'd advise applying jitter to the matrix on CPU, and then passing the jitter offsets in as uniforms for current and previous frames jitter offsets. It doesn't make sense to repeatedly calculate the same jitter in a shader. Do it once per frame on the CPU.

@Aressera I finally got around to implementing it as you mentioned. I am now creating the jitter and applying it to the matrix in the CPU and passing it to the GPU. I'm also passing the current jitter as well as the previous jitter as uniforms as well. My new velocity calculations looks like this.

void Velocity(uint3 DTid : SV_DispatchThreadID)
{
	float2 texelSize = float2(1.0f / 1920, 1.0f / 1080);
	float2 texCoords = texelSize * (DTid.xy + 0.5);

	// Calculate current position world space
	float depth = other.SampleLevel(sPointClamp, texCoords, 0).x;
	float ndcX = texCoords.x * 2.0 - 1.0;
	float ndcY = 1.0f - texCoords.y * 2.0f;
	float4 viewpos = mul(float4(ndcX, ndcY, depth, 1.0), InverseProjectionMatrix);
	viewpos = viewpos / viewpos.w;
	float4 worldspace = mul(float4(viewpos.xyz, 1.0), InverseViewMatrix);

	// Reproject using previous view and proj matrix
	float4x4 prevViewProj = mul(prevView, prevProj);
	float4 previousPosNDC = mul(float4(worldspace.xyz, 1.0), prevView);
	previousPosNDC = mul(float4(previousPosNDC.xyz, 1.0), prevProj);
	previousPosNDC.xy /= previousPosNDC.w;
	float2 prevPositionSS = (previousPosNDC.xy * float2(0.5, -0.5) + float2(0.5, 0.5));

	// Calculate velocity: current position in SS minus previous position SS
	float4 ssPos = float4(texCoords.x, texCoords.y, depth, 1.0);

	// Remove jitter from both ss points

	float2 remove = (currentoffset - prevoffset);

	float2 velocity = ssPos.xy - prevPositionSS;
	velocity -= remove;

	dst[DTid.xy] = float4(velocity, 0.0, 1.0);
}

Despite doing this, my velocity texture is still outputting black and no actual velocity values.

@Aressera I think I might be doing something wrong and wanted to ask. When we take the current screen-space position and convert it back to World space, we would need to use the inverse of the projection matrix. However, would this inverse projection matrix be the normal unjittered projection matrix or the projection matrix we applied jitter to?

I'm not sure, but I would probably use the jittered matrix to unproject the depth. I'm not sure if your approach of using scene depth is the right way to calculate velocity. The examples I've seen do the work in the vertex shader of the main pass (gbuffer/forward), where the vertices get multiplied by model-view-projection matrices from current and previous frames to get screen space positions. Disclaimer: I haven't implemented any of this.

I have read the following good tutorials and they may answer your questions:

https://www.elopezr.com/temporal-aa-and-the-quest-for-the-holy-trail/​

https://alextardif.com/TAA.html