⚖️ Load Balancer
Request distribution policies
Running
Policy: Round-robin
Policy
Traffic
Controls
Stats
Avg latency
0.0
Completed
0
In system
0
Dropped
0
Per server
Info & Theory

A load balancer spreads incoming requests across a pool of backend servers so none is overwhelmed while others idle. The policy decides which server gets each request.

The policies

  • Round-robin — cycle through servers in order.
  • Least-connections — send to the server with the fewest active requests.
  • Weighted — give faster servers proportionally more traffic.
  • Random — pick a server uniformly at random.

Poisson arrivals

Requests arrive randomly: inter-arrival gaps are −ln(U)/λ for random U, the classic Poisson process seen in real bursty traffic.

Utilisation and latency

Utilisation ρ = λ/μ is the busy fraction of a server. Queueing theory gives waiting time ∝ 1/(1 − ρ), so latency explodes as ρ → 1. Round-robin can still pile queues onto slow servers; least-connections adapts to keep them even.

Frequently asked questions

What does a load balancer do?

A load balancer sits in front of a pool of backend servers and decides which server should handle each incoming request, spreading traffic so no single server is overwhelmed while others sit idle.

How does round-robin balancing work?

Round-robin sends each successive request to the next server in a fixed cyclic order: server 1, server 2, server 3, then back to 1. It is simple and fair when servers are identical and requests cost roughly the same.

What is least-connections balancing?

Least-connections sends each new request to the server currently handling the fewest active requests. It adapts to uneven request durations and unequal server speeds far better than round-robin.

What is weighted load balancing?

Weighted balancing assigns each server a weight proportional to its capacity, so faster servers receive proportionally more requests. A server with weight 3 gets roughly three times the share of one with weight 1.

When is random load balancing acceptable?

Picking a server uniformly at random is stateless and trivial to implement, and with many servers it approaches even distribution. The "power of two random choices" variant — pick two, take the less loaded — performs remarkably well.

What are Poisson arrivals?

Poisson arrivals model independent random request times where the gaps between arrivals follow an exponential distribution. This simulation generates them with inter-arrival time −ln(U)/λ for a random U, matching real bursty traffic.

Why does latency rise sharply near full utilisation?

Queueing theory shows waiting time grows like 1/(1 − ρ), where ρ is utilisation. As a server approaches 100% busy, even a small traffic increase causes queues and latency to explode.

What is server utilisation?

Utilisation ρ is the fraction of time a server is busy, equal to arrival rate divided by service rate for that server. Above ρ = 1 the server cannot keep up and its queue grows without bound.

Why can round-robin still cause imbalance?

Round-robin ignores how long each request takes and how fast each server is. If requests vary in cost or servers differ in speed, some servers build up queues while others drain, even though counts are equal.

How is average latency measured here?

The simulation records each completed request's total time — waiting in the queue plus being served — and reports the running average across all finished requests under the current policy.

About the Load Balancer Simulation

Written by MySimulator Team · Reviewed by MySimulator Editorial Review

Last updated: 5 July 2026

This simulation models a load balancer distributing incoming requests across a pool of backend servers. Requests arrive as a Poisson process, with inter-arrival gaps drawn from an exponential distribution using the formula minus the natural log of a uniform random number, divided by the arrival rate lambda. Each server handles one request at a time, with its own exponentially distributed service duration set by an individual service rate mu, while waiting requests sit in a per-server queue.

The policy buttons switch between round-robin, least-connections, weighted and random dispatch, and the sliders adjust the number of servers (2 to 8), the arrival rate lambda (0.5 to 9) and the simulation speed. The animation shows requests travelling from the LB node to their chosen server, queue depth as dots, and live statistics for average latency, completed requests, requests in system and dropped requests. It illustrates why queueing systems become unstable as utilisation approaches one, a core concern in real distributed systems.

Frequently Asked Questions

What is a load balancer and what does this simulation show?

A load balancer sits in front of several backend servers and decides which one should handle each incoming request. This simulation animates that decision in real time, showing requests flowing from the central LB node to servers under four different dispatch policies, with live queues, latency and utilisation so you can compare how the policies behave.

How does the round-robin policy work here?

Round-robin keeps a pointer that cycles through the servers in fixed order: server 0, then 1, then 2, and back to 0. It is fair by request count and trivial to implement, but it ignores how busy or how fast each server is, so it can still pile queues onto slower servers.

What is the difference between least-connections and weighted dispatch?

Least-connections sends each new request to the server with the fewest active requests, counting both its current job and its queue, so it adapts to uneven load. Weighted dispatch instead picks a server with probability proportional to its weight, which in this model equals its service rate, so faster servers receive proportionally more traffic regardless of current load.

What do the Servers, Arrival rate and Sim speed sliders control?

The Servers slider sets how many backend servers exist, between 2 and 8. The Arrival rate slider sets lambda, the average number of requests per simulated second, from 0.5 to 9. The Sim speed slider scales how fast simulated time advances relative to real time, letting you watch slowly or fast-forward, without changing the underlying statistics.

What are Poisson arrivals and why are they used?

A Poisson process models independent random events at a steady average rate, where the gaps between arrivals follow an exponential distribution. The simulation generates each gap as minus the natural log of a uniform random number divided by lambda. Real web and network traffic is bursty and well approximated by this model, which is why it is the standard assumption in queueing theory.

Why does latency explode as utilisation nears 100 percent?

Utilisation rho is the fraction of time a server is busy, equal to its arrival rate divided by its service rate mu. Classic queueing results show average waiting time grows in proportion to one over one minus rho. As rho approaches one, that term diverges, so small increases in traffic cause queues and latency to rise sharply, and beyond rho equal to one the queue grows without bound.

What does the Dropped counter mean?

Each server has a maximum queue length of twelve. If a request arrives at a server whose queue is already full and which is currently busy, it cannot be admitted and is counted as dropped rather than queued. A rising Dropped count signals that the system is overloaded for the chosen policy and arrival rate.

How is average latency calculated in this model?

When a request finishes service, the simulation takes its total time in the system, which is the current simulated time minus the time it was born, covering both queue waiting and service. These totals are summed and divided by the number of completed requests, giving the running average latency shown in the statistics panel.

Is this simulation physically and mathematically accurate?

It uses genuinely exponential inter-arrival and service times, single-server FIFO queues and the standard four dispatch policies, so it faithfully reproduces the qualitative behaviour of M/M/c-style queueing systems. Server service rates are assigned in a fixed pattern rather than randomly, and the queue cap is finite, so it is a faithful teaching model rather than a production performance predictor.

Why can round-robin still cause imbalance even though counts are equal?

Round-robin equalises the number of requests each server receives, but not the work. If requests vary in service time or servers differ in speed mu, a slower server given the same count of jobs falls behind, building a long queue while faster servers drain to idle. Least-connections and weighted policies address this by reacting to load or capacity.

Where are these load balancing policies used in the real world?

They underpin web server farms, content delivery networks, database read replicas and microservice meshes. Reverse proxies such as NGINX and HAProxy and cloud load balancers offer round-robin, least-connections and weighted modes directly. The simulation captures the same trade-offs engineers weigh when choosing a policy for production traffic.