📊 Data Quality Score Pipeline

About this simulation

Most machine learning effort is spent not on algorithms but on preparing data. This scene visualises that work as a physical pipeline: raw records leave a "data lake" and pass through five quality gates — completeness, accuracy, consistency, timeliness and uniqueness — the five components of a typical Data Quality Score (DQS). Each record's composite score then decides which storage tier it lands in.

How it works

• Every record is spawned with five random dimension scores; a higher defect rate drags all five down, simulating messier source systems.
• Each gate ring flashes green when a record passes that dimension (≥ 50) and red when it fails.
• The composite DQS is the average of all five dimensions, scaled to 0–100.
• Records below 40 are quarantined; 40 up to the gold threshold go to the silver tier (cleaned, validated); at or above the threshold they reach the gold tier (curated, ML-ready).

Controls

• Defect rate — how dirty the incoming source data is (missing fields, typos, duplicates, stale timestamps).
• Ingestion rate — how many records per second enter the pipeline.
• Gold threshold — the DQS cutoff a record must clear to reach the gold, ML-ready tier.
• Show quarantined records — toggle whether failed records are shown falling into the quarantine bin.
• ↺ Reset pipeline — clears all in-flight records and stats. Drag to rotate, scroll to zoom.

Did you know?

Data teams commonly report spending 60–80% of a machine learning project's time on cleaning, validating and re-shaping data — which is exactly why a tiered bronze/silver/gold storage architecture and an explicit quality score exist: they make "is this data trustworthy enough to train on?" a measurable question.