A hidden 150×108 canvas renders the chosen synthetic video (dots, checkerboard or bars, panning / rotating / zooming, or a moving disc over a static field) — that pixel buffer is the real input, not a decoration. Every frame the engine takes the grayscale image, computes spatial gradients Iₓ, Iᵧ by central differences and a temporal gradient Iₜ = I(t) − I(t−1), then solves the Lucas–Kanade normal equations over a small window at each sample point:
[ΣIx² ΣIxIy ] [u] [ΣIxIt]
[ΣIxIy ΣIy² ] [v] = -[ΣIyIt]
λmin = confidence (Shi–Tomasi)
The 2×2 "structure tensor" on the left is solved by Cramer's rule for the per-point velocity (u, v). Its smaller eigenvalue λmin is the classic Shi–Tomasi corner score: where the local patch has gradient in only one direction (a stripe, an edge), λmin collapses to ~0 and the two equations become linearly dependent — the true 2-D motion can't be recovered, only the component perpendicular to the edge. That's the aperture problem: switch the pattern to Bars and watch confident (green) points thin out to whichever moments the sweep direction crosses the stripes squarely.
- Pattern — dots and checker corners give a well-conditioned tensor almost everywhere; bars are mostly one-directional gradient, so confidence swings with the current motion direction.
- Motion — Pan sweeps a slowly rotating direction; Rotate and Zoom generate the swirling / radiating flow fields real cameras see from platform yaw and forward travel (the zoom case is a focus-of-expansion, the field autonomous-driving pipelines use to estimate time-to-collision); Object keeps the background static so only the moving disc lights up — real motion segmentation.
- Analysis window — the summation radius in pixels; a bigger window averages more gradient evidence (fewer false aperture readings, coarser localisation), a smaller one is sharper but noisier.
Vectors are drawn from each sample point, amplified ×8 for visibility (raw per-frame pixel displacement here is well under one pixel); dot colour marks whether that point passed the confidence test.