Home▸Data Science▸Kafka Partition Log & Consumer Lag Simulator (2D)

Kafka Partition Log & Consumer Lag Simulator (2D)

A 2D companion to the 3D Kafka visualizer, rendered as a top-down partitioned topic log instead of an orbitable scene: producers append events onto each partition's log at an offset, two independent consumers replay the same log at their own pace, and a lagging consumer that falls outside the retention window gets a real offset out-of-range reset — with a live lag-over-time strip chart making the failure mode directly readable.

Data Science2DModerate60 FPS📱 Mobile-adapted⇄ 3D version

This is a genuine, independently-built 2D counterpart to the 3D Kafka visualizer — not a reprojection of the orbitable scene. Each partition is drawn as a horizontal lane with events rendered left-to-right by offset, the newest event always at the head on the right. A green flag tracks Consumer A, which stays close to real time; an amber flag tracks Consumer B, whose processing speed you control directly. Push producer rate above Consumer B's speed and watch its flag fall behind the dashed retention boundary — cross that line and Kafka can no longer serve the requested offset, forcing a real offset out-of-range reset (a red flash, and the consumer jumps forward to the oldest offset still available). A strip chart along the bottom plots Consumer B's lag over time against the retention limit, turning a subtle streaming failure mode into something you can watch happen and immediately correlate with the rate and retention sliders that caused it.

⚙ Under the hood

2D companion to the 3D Kafka topic-log visualizer: partitions as horizontal lanes with offset-indexed events, a real-time Consumer A and a speed-limited Consumer B, retention-window eviction, and a genuine offset out-of-range reset when Consumer B's offset falls outside the retention window — plotted live on a lag-over-time strip chart.

apache-kafkaevent-driven-architecturemessage-queuesdata-streamingdistributed-systemspub-subdata-science

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install