Why the Algorithm Choice Actually Matters
A load balancer is only as good as its scheduling logic. Pick the wrong algorithm and you'll see one backend saturated while others sit idle, sticky sessions breaking at the worst moment, or cache hit rates cratering after a routine deployment. The three algorithms you'll encounter most often are round-robin, least-connections, and hashing. Here's how they work and when to reach for each.
Round-Robin
Round-robin distributes incoming requests to backends in a fixed rotation: request 1 goes to server A, request 2 to server B, request 3 to server C, then back to A, and so on.
How it works
Most implementations support a weighted variant. Assign server A a weight of 2 and server B a weight of 1, and A receives twice as many requests—useful when your nodes have unequal capacity.
When it fits
- Stateless services where every request is roughly the same cost (static asset servers, simple REST APIs)
- Homogeneous backend fleets where all nodes have identical specs
- Situations where simplicity and predictability matter more than optimal distribution
Where it breaks
Round-robin is blind to backend state. If one request takes 50 ms and another takes 5 seconds, round-robin still sends the next request to whichever node is "next" regardless of how busy it already is. Under variable request latency this creates hot spots.
Least-Connections
Least-connections routes each new request to the backend with the fewest active connections at that moment.
How it works
The load balancer maintains a live connection count per backend. When a request arrives it picks the minimum. Some implementations use least response time as a refinement—choosing the backend with the fewest connections and the lowest average latency.
When it fits
- Long-lived connections (WebSockets, database proxies, streaming)
- Workloads with high variance in request duration
- Mixed fleets where some requests are cheap and some are expensive
Where it breaks
Least-connections adds state the load balancer must track. In high-throughput, short-lived HTTP/1.1 scenarios, connection counts can update so fast that the "minimum" you picked is already stale by the time the packet arrives. It also doesn't help if your bottleneck is CPU or memory rather than connection count—a backend with two long-running CPU-intensive jobs looks cheaper than one handling four idle keep-alive connections.
Hashing
Hashing computes a hash of some property of the request—most commonly the client IP, a URL, or a session cookie—and maps the result to a specific backend.
How it works
The naive version hashes directly to a backend index. Consistent hashing is the production-grade variant: backends sit on a virtual ring, and adding or removing a node only remaps a fraction of the keyspace rather than reshuffling everything. This is the algorithm behind most distributed caches and is used natively in systems like Nginx's hash directive and HAProxy's balance uri.
When it fits
- Caching layers where you want the same URL to always hit the same backend (maximizes cache hit rate)
- Services with server-side session state that hasn't been moved to a shared store
- Any use case where request affinity is a correctness requirement, not just a preference
Where it breaks
Hash keys with low cardinality can concentrate traffic on a few backends. Health checks determine when failed targets leave the eligible pool. External monitoring observes the public result; it does not replace the balancer's health checks or guarantee detection before users see errors.
Choosing in Practice
Use this as a starting decision tree:
- Do requests require affinity to a specific backend? → Consistent hashing
- Are request durations highly variable or connections long-lived? → Least-connections
- Is your fleet homogeneous and your requests roughly uniform? → Round-robin (weighted if needed)
- Are you unsure? → Start with least-connections. It degrades more gracefully than round-robin under load spikes and is less sensitive to keyspace skew than hashing.
In practice, many teams layer algorithms: consistent hashing at the edge for cache affinity, then least-connections inside the origin tier where request cost varies.
Key Takeaways
- Round-robin is simple and predictable but blind to backend state—only use it when request cost is uniform.
- Least-connections adapts to variable workloads but requires the load balancer to track live state.
- Consistent hashing is essential for cache affinity but can create hot spots with low-cardinality keys.
- Weighted variants of round-robin and least-connections help when backends aren't identical.
- The gap between a backend failing and your load balancer noticing matters—external monitoring closes that gap.
- No single algorithm is universally best; production systems often combine them across tiers.