Cloud Load Balancer & Auto-Scaling Architecture

Traffic

Request rate 8 req/s

Load-balancing algorithm

Scale-out CPU threshold 70%

Fleet Status

Active instances

0 / 15

Avg CPU load

Throughput

0 req/s

Dropped

Availability Zones

How it works

Requests spawn from the client at the chosen rate and are routed by the load balancer to one of the healthy instances across three availability zones (AZs):

Round Robin:       pick the next instance in a fixed rotation
Least Connections:  pick the instance with the fewest in-flight requests
Random:              pick uniformly among healthy, non-full instances

Each instance can process up to 4 concurrent requests. An instance's CPU load is modelled as load = in_flight / 4. Every second the fleet-wide average CPU is compared against the scale-out threshold:

avgCPU > scaleOutThreshold  → spin up a new instance (in the least-loaded AZ)
avgCPU < scaleOutThreshold − 30%  → spin down an idle instance
a 2s cooldown after each action prevents oscillation ("flapping")

If every healthy instance is already at its concurrency limit, an incoming request is dropped rather than queued indefinitely — visible as the Dropped counter and a red flash at the load balancer. Simulate AZ Outage marks the middle zone unhealthy: the balancer instantly stops routing to it and the surviving zones absorb the traffic, demonstrating the failover a real multi-AZ deployment relies on.

Request in flight Healthy instance Failed AZ / dropped

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.