Temporal Anti-Aliasing (TAA) Explained

1. The aliasing problem

A triangle edge is a perfectly sharp mathematical line, but a pixel is a square sampling area. When an edge crosses a pixel, the renderer must decide: is this pixel "inside" the triangle or "outside"? A single point sample at the pixel centre gives a binary answer, which produces the jagged staircase pattern known as aliasing.

The classic fix, MSAA (Multisample Anti-Aliasing), takes multiple coverage samples per pixel within a single frame — 4x or 8x — and averages them. It's accurate but expensive: 8x MSAA means roughly 8x the geometry coverage tests and a much larger G-buffer, and it does nothing for aliasing inside a shader (specular sparkle, alpha-tested foliage).

2. Sub-pixel jitter

Each frame, the camera's projection matrix is offset by a tiny sub-pixel amount — the jitter — following a low-discrepancy sequence (Halton 2,3 is standard) that covers the pixel footprint evenly over N frames without repeating patterns that the eye could pick up on.

This offset is added directly into the projection matrix before rendering, so every triangle edge, shadow sample and specular highlight is evaluated at a slightly different sub-pixel position each frame — exactly what MSAA does spatially within one frame, TAA does temporally across many.

3. History reprojection with motion vectors

Blending this frame with last frame's colour buffer only works if you're blending the same surface point. If the camera or an object moved, the surface point that was at pixel (400, 300) last frame might be at (410, 295) this frame. TAA fixes this with a motion vector buffer — for every pixel, the screen-space displacement between this frame's position and last frame's position of the same world point:

The TAA resolve pass then samples the history buffer at currentUV − motionVector instead of currentUV — this is reprojection. For static geometry the camera's own motion is enough; for skinned or animated meshes the engine must write per-vertex velocity into the motion vector buffer explicitly.

4. Neighbourhood clamping

Even with correct motion vectors, disocclusion (new geometry that wasn't visible last frame — think of a wall edge revealing what was behind an object) has no valid history sample. Blending in garbage history there would produce smearing. The standard fix is neighbourhood clamping: build an axis-aligned bounding box (AABB) — or a tighter variance-based ellipsoid — from the current frame's 3×3 pixel neighbourhood in colour space, and clamp the reprojected history sample into that box before blending.

This forces the history colour to stay plausible relative to what the current frame actually sees, at the cost of slightly reducing the effective temporal sample count near hard edges and disocclusions — a necessary trade-off, since unclamped history would ghost visibly.

5. Ghosting and disocclusion

Ghosting is the visible trailing artifact left behind fast-moving thin objects (a spinning propeller, chain-link fence, or particle effects) — the classic TAA complaint. It happens when neighbourhood clamping can't fully reject invalid history because the box built from a thin, high-frequency neighbourhood is itself unreliable.

• Velocity-weighted blend factor: reduce the history blend weight (normally ~0.9) when motion vector magnitude is large, favouring the current frame more for fast-moving pixels.
• Depth-based rejection: reject history if the reprojected depth doesn't match the current depth within a threshold — a strong signal of disocclusion.
• Sharper jitter patterns + tonemap-aware blending: blend in a perceptual (tonemapped) colour space so bright, high-contrast highlights don't dominate the variance clip box.

6. A minimal TAA resolve pass

A simplified fragment shader for the resolve step:

uniform sampler2D uCurrentColor;
uniform sampler2D uHistoryColor;
uniform sampler2D uMotionVectors;
uniform float uBlendFactor; // ~0.9 static, lower under fast motion

vec3 clipAABB(vec3 aabbMin, vec3 aabbMax, vec3 historyColor) {
  vec3 center  = 0.5 * (aabbMax + aabbMin);
  vec3 extents = 0.5 * (aabbMax - aabbMin) + 0.0001;
  vec3 offset  = historyColor - center;
  vec3 unitOffset = abs(offset / extents);
  float maxUnit = max(unitOffset.x, max(unitOffset.y, unitOffset.z));
  return maxUnit > 1.0 ? center + offset / maxUnit : historyColor;
}

void main() {
  vec2 motion  = texture(uMotionVectors, vUv).xy;
  vec2 prevUv  = vUv - motion;

  vec3 current = texture(uCurrentColor, vUv).rgb;
  vec3 history = texture(uHistoryColor, prevUv).rgb;

  // Build 3x3 neighbourhood AABB from current frame
  vec3 aabbMin = current, aabbMax = current;
  for (int x = -1; x <= 1; x++)
  for (int y = -1; y <= 1; y++) {
    vec3 s = textureOffset(uCurrentColor, vUv, ivec2(x, y)).rgb;
    aabbMin = min(aabbMin, s);
    aabbMax = max(aabbMax, s);
  }

  history = clipAABB(aabbMin, aabbMax, history);

  bool validHistory = all(lessThan(abs(prevUv - 0.5), vec2(0.5)));
  float blend = validHistory ? uBlendFactor : 0.0;
  fragColor = vec4(mix(current, history, blend), 1.0);
}

7. TAA vs. MSAA vs. FXAA vs. DLSS

• MSAA: highest quality edge AA, no ghosting, but expensive and blind to shader/specular aliasing; incompatible with deferred rendering without extra G-buffer work.
• FXAA: a single-pass post-process edge blur — cheap, no jitter or history needed, but can blur fine detail and text since it has no sub-pixel information to work with.
• TAA: cheap per-frame, handles shader aliasing, but introduces ghosting risk and a slight softness that needs sharpening to counteract.
• DLSS / FSR / TAAU (temporal upscaling): extend TAA's reprojection machinery to also upscale — render at a lower internal resolution and let the temporal accumulation reconstruct detail at output resolution, using a trained neural network (DLSS) or hand-tuned heuristics (FSR2) instead of a simple AABB clamp.