Throughput = min(ingest_rate, transform_capacity)
Latency ≈ Σ stage_processing_time
Queue(t+dt) = Queue(t) + (in − out)·dt
A classic ETL (Extract-Transform-Load) pipeline moves records through discrete stages: Source → Extract → Transform → Load → Warehouse. Each stage has finite throughput; when the transform stage is slower than ingestion, records queue up (backpressure), increasing end-to-end latency - exactly what happens in real streaming systems like Kafka/Spark pipelines.
- Ingest rate — records per second arriving from the source system.
- Transform complexity — how much processing each record needs; higher values slow the transform stage's throughput.
- Error rate — percentage of records that fail validation and are routed to a dead-letter queue instead of the warehouse.
- Queue backlog — toggles a visible stack of packets waiting at the bottleneck stage.
Real-world application: data engineers monitor exactly these metrics (throughput, latency, error/dead-letter rate) in tools like Airflow, dbt, and Kafka to keep data warehouses fresh and reliable.