Serverless Cold Start Latency Simulator

Traffic & Scaling

Request rate 6.0 req/s

Cold start latency 900 ms

Idle timeout (scale-down) 15 s

Concurrency / instance 4

Live Metrics

Active instances

0 / 30

Cold-start rate

p50 latency

0 ms

p95 latency

0 ms

Requests served

Queued (backpressure)

How it works

This models a serverless / FaaS autoscaler as an M/G/c-style queue with a warm pool. Each function instance can serve up to c concurrent requests. When a request arrives:

If a warm instance has spare concurrency, it is dispatched immediately (near-zero latency).
If none are available but capacity remains, a new instance is provisioned — it must finish a fixed cold-start delay T_cold before it can execute, so that request's latency is at least T_cold.
If every instance is saturated, the request queues until a slot frees up (backpressure), adding further delay.

latency = T_cold · [instance was cold] + T_queue + T_exec
p50 / p95 computed over the last 200 completed requests

An idle instance with zero active requests for longer than the idle timeout is torn down (scaled to zero), which is why bursty traffic after a quiet period pays the cold-start tax again — the classic serverless latency/cost trade-off used by AWS Lambda, Azure Functions and Google Cloud Run.

Off (scaled to zero) Cold starting Warm / idle Busy (serving)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.