What does a load balancer do?

A load balancer sits in front of a pool of backend servers and decides which server should handle each incoming request, spreading traffic so no single server is overwhelmed while others sit idle.

How does round-robin balancing work?

Round-robin sends each successive request to the next server in a fixed cyclic order: server 1, server 2, server 3, then back to 1. It is simple and fair when servers are identical and requests cost roughly the same.

What is least-connections balancing?

Least-connections sends each new request to the server currently handling the fewest active requests. It adapts to uneven request durations and unequal server speeds far better than round-robin.

What is weighted load balancing?

Weighted balancing assigns each server a weight proportional to its capacity, so faster servers receive proportionally more requests. A server with weight 3 gets roughly three times the share of one with weight 1.

When is random load balancing acceptable?

Picking a server uniformly at random is stateless and trivial to implement, and with many servers it approaches even distribution. The 'power of two random choices' variant — pick two, take the less loaded — performs remarkably well.

Why does latency rise sharply near full utilisation?

Queueing theory shows waiting time grows like 1/(1 − ρ), where ρ is utilisation. As a server approaches 100% busy, even a small traffic increase causes queues and latency to explode.

What is server utilisation?

Utilisation ρ is the fraction of time a server is busy, equal to arrival rate divided by service rate for that server. Above ρ = 1 the server cannot keep up and its queue grows without bound.

How is average latency measured here?

The simulation records each completed request's total time — waiting in the queue plus being served — and reports the running average across all finished requests under the current policy.

Home▸Distributed & Parallel Computing▸Load Balancer — Request Distribution Policies

⚖️ Load Balancer — Request Distribution Policies

Route incoming requests across backend servers with round-robin, least-connections, weighted and random policies. Watch queues, latency and utilisation diverge as traffic and server speeds vary.

Distributed & Parallel Computing3DEasy60 FPS

About the Load Balancer Simulation

This simulation models a load balancer distributing incoming requests across a pool of backend servers. Requests arrive as a Poisson process, with inter-arrival gaps drawn from an exponential distribution using the formula minus the natural log of a uniform random number, divided by the arrival rate lambda. Each server handles one request at a time, with its own exponentially distributed service duration set by an individual service rate mu, while waiting requests sit in a per-server queue.

The policy buttons switch between round-robin, least-connections, weighted and random dispatch, and the sliders adjust the number of servers (2 to 8), the arrival rate lambda (0.5 to 9) and the simulation speed. The animation shows requests travelling from the LB node to their chosen server, queue depth as dots, and live statistics for average latency, completed requests, requests in system and dropped requests. It illustrates why queueing systems become unstable as utilisation approaches one, a core concern in real distributed systems.

Frequently Asked Questions

What is a load balancer and what does this simulation show?

A load balancer sits in front of several backend servers and decides which one should handle each incoming request. This simulation animates that decision in real time, showing requests flowing from the central LB node to servers under four different dispatch policies, with live queues, latency and utilisation so you can compare how the policies behave.

How does the round-robin policy work here?

Round-robin keeps a pointer that cycles through the servers in fixed order: server 0, then 1, then 2, and back to 0. It is fair by request count and trivial to implement, but it ignores how busy or how fast each server is, so it can still pile queues onto slower servers.

What is the difference between least-connections and weighted dispatch?

Least-connections sends each new request to the server with the fewest active requests, counting both its current job and its queue, so it adapts to uneven load. Weighted dispatch instead picks a server with probability proportional to its weight, which in this model equals its service rate, so faster servers receive proportionally more traffic regardless of current load.

What do the Servers, Arrival rate and Sim speed sliders control?

The Servers slider sets how many backend servers exist, between 2 and 8. The Arrival rate slider sets lambda, the average number of requests per simulated second, from 0.5 to 9. The Sim speed slider scales how fast simulated time advances relative to real time, letting you watch slowly or fast-forward, without changing the underlying statistics.

What are Poisson arrivals and why are they used?

A Poisson process models independent random events at a steady average rate, where the gaps between arrivals follow an exponential distribution. The simulation generates each gap as minus the natural log of a uniform random number divided by lambda. Real web and network traffic is bursty and well approximated by this model, which is why it is the standard assumption in queueing theory.

Why does latency explode as utilisation nears 100 percent?

Utilisation rho is the fraction of time a server is busy, equal to its arrival rate divided by its service rate mu. Classic queueing results show average waiting time grows in proportion to one over one minus rho. As rho approaches one, that term diverges, so small increases in traffic cause queues and latency to rise sharply, and beyond rho equal to one the queue grows without bound.

What does the Dropped counter mean?

Each server has a maximum queue length of twelve. If a request arrives at a server whose queue is already full and which is currently busy, it cannot be admitted and is counted as dropped rather than queued. A rising Dropped count signals that the system is overloaded for the chosen policy and arrival rate.

How is average latency calculated in this model?

When a request finishes service, the simulation takes its total time in the system, which is the current simulated time minus the time it was born, covering both queue waiting and service. These totals are summed and divided by the number of completed requests, giving the running average latency shown in the statistics panel.

Is this simulation physically and mathematically accurate?

It uses genuinely exponential inter-arrival and service times, single-server FIFO queues and the standard four dispatch policies, so it faithfully reproduces the qualitative behaviour of M/M/c-style queueing systems. Server service rates are assigned in a fixed pattern rather than randomly, and the queue cap is finite, so it is a faithful teaching model rather than a production performance predictor.

Why can round-robin still cause imbalance even though counts are equal?

Round-robin equalises the number of requests each server receives, but not the work. If requests vary in service time or servers differ in speed mu, a slower server given the same count of jobs falls behind, building a long queue while faster servers drain to idle. Least-connections and weighted policies address this by reacting to load or capacity.

Where are these load balancing policies used in the real world?

They underpin web server farms, content delivery networks, database read replicas and microservice meshes. Reverse proxies such as NGINX and HAProxy and cloud load balancers offer round-robin, least-connections and weighted modes directly. The simulation captures the same trade-offs engineers weigh when choosing a policy for production traffic.

⚙ Under the hood

Route incoming requests across backend servers with round-robin, least-connections, weighted and random policies. Watch queues, latency and utilisation diverge as traffic and server speeds vary.

load balancinground robinleast connectionsqueueingCanvas 2D

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install