Mobile Test Pyramid: Coverage vs. Cost vs. Flakiness

Test Suite Shape

Unit tests 400

Integration tests 60

UI / E2E tests 15

Parallel CI workers 8

Adjust the pyramid, then run the pipeline.

Live Pipeline Stats

Wall-clock time

—

Defect escape prob.

—

Expected flaky fails/run

—

P(≥1 flaky red build)

—

How it works

Three tiers run in sequence, each parallelised across the same worker pool. Stage wall-clock time uses bin-packing over workers:

T_stage = ceil(n / workers) × avg_duration
T_total = T_unit + T_integration + T_ui

avg_duration: unit 0.05s, integration 1.2s, UI 8s

Each tier is only good at catching certain defect classes. Detection probability saturates with diminishing returns as a tier grows (n_half = tests needed for ~63% catch rate):

p_tier(n) = 1 − e^(−n / n_half)
n_half: unit 150, integration 40, UI 8

P(escape) = 0.70(1−p_unit) + 0.20(1−p_integration) + 0.10(1−p_ui)

The 70/20/10 split is the classic testing-pyramid claim: most real regressions are narrow logic bugs a unit test can catch; progressively fewer need integration or full end-to-end context to surface.

Flaky (non-deterministic) tests fail at random even on green code. Per-test flake rates: unit 0.02%, integration 0.3%, UI 1.5% — UI tests touch real timing, animation and network, so they flake far more often. The count of falsely-red tests in one run is Poisson-distributed:

λ = n_unit·0.0002 + n_int·0.003 + n_ui·0.015   (expected flaky fails)
P(≥1 flaky red build) = 1 − e^(−λ)

Run CI Pipeline fires one stochastic run: cubes light green (pass), red (flaky false-fail) tier by tier, then a bug marker is rolled against the escape probability and either gets caught by a tier or flies off the top of the pyramid into production.

Unit tier (base) Integration tier UI / E2E tier (top) Flaky failure

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.