Why the Algorithm Matters
A load balancer is only as good as its distribution logic. Pick the wrong algorithm and you'll see hot spots, session breakage, or uneven resource drain even when you have plenty of capacity. The three algorithms you'll encounter most are round-robin, least-connections, and hashing. Each solves a different problem.
Round-Robin
Round-robin rotates requests across backends in a fixed sequence: request 1 goes to server A, request 2 to server B, request 3 to server C, then back to A.
How it works
Most implementations support a weighted variant. Assign server A a weight of 2 and server B a weight of 1, and A receives roughly twice as many requests. NGINX's upstream block uses weighted round-robin by default.
When it fits
- Stateless services where every backend can handle every request equally
- Homogeneous hardware where processing time per request is roughly uniform
- Simple deployments where you want predictable, auditable distribution
Where it breaks down
Round-robin ignores how long requests actually take. If some requests are fast (a cache hit) and others are slow (a database write), backends can pile up work unevenly. One slow request doesn't block the next assignment—the balancer just keeps rotating.
Least-Connections
Instead of rotating blindly, the balancer tracks the active connection count on each backend and always forwards to the one with the fewest open connections.
How it works
HAProxy calls this leastconn. NGINX offers least_conn. The balancer maintains a counter per backend, increments on connection open, decrements on close. Ties are broken by weight or position.
When it fits
- Long-lived connections: WebSockets, gRPC streams, database proxies
- Heterogeneous request durations where some requests are significantly slower than others
- Backends that vary in speed due to garbage collection, query complexity, or external dependencies
Where it breaks down
Least-connections works on count, not cost. A backend handling 10 lightweight polling requests looks busier than one handling 2 heavy report-generation jobs. For CPU-bound workloads, consider least-response-time if your load balancer supports it (HAProxy's leastconn with slowstart, or NGINX Plus's least_time).
It also adds a small overhead: the balancer must read and update shared counters, which matters at very high request rates.
Hashing
Hashing routes requests deterministically based on some attribute—client IP, a header value, a URL parameter, or a cookie—so the same input always maps to the same backend.
How it works
The balancer computes a hash of the chosen key and maps it to a backend slot. Consistent hashing (used by systems like Envoy and many CDNs) minimises redistribution when backends are added or removed by arranging nodes on a ring.
When it fits
- Stateful applications where session data lives on a specific backend and you can't or won't use a shared session store
- Cache servers where you want the same content type to hit the same node to maximise cache hit rate
- APIs with rate limiting per client where you want affinity to simplify counter storage
Where it breaks down
Hashing can create hot spots if a small number of clients generate a disproportionate share of traffic (think a single high-volume API consumer). It also makes rolling deployments trickier: adding or removing a backend redistributes some requests, potentially invalidating cache entries or dropping sessions unless you use consistent hashing.
Choosing the Right Algorithm
Here's a quick decision path:
- Are your requests stateless and roughly uniform in cost? → Start with round-robin (weighted if backends differ in capacity).
- Do you have long-lived or variable-cost connections? → Use least-connections.
- Do you need client or content affinity? → Use hashing, preferably consistent hashing.
- Is your backend pool dynamic (autoscaling)? → Avoid plain modulo hashing; consistent hashing handles node churn far better.
- Do you have very high throughput and need response-time awareness? → Evaluate least-response-time if available.
Monitoring Validates Your Choice
No algorithm choice is fire-and-forget. Backend response times, error rates, and connection counts should be watched continuously. If you're running multi-region infrastructure, external uptime monitoring from locations outside your own network (like Pingy) will surface whether a misconfigured algorithm is causing failures that internal health checks miss—for example, a backend that's reachable internally but timing out for real users in a specific region.
Set up latency alerts per origin, not just aggregate uptime, so you can correlate spikes with deployment or scaling events.
Key Takeaways
- Round-robin is simple and predictable; use it for stateless, uniform workloads.
- Least-connections adapts to variable request cost; essential for long-lived or slow connections.
- Hashing provides affinity and is good for caching layers, but watch for hot spots and use consistent hashing in dynamic environments.
- Algorithm choice interacts with your session storage, caching strategy, and deployment process—test changes before rolling to production.
- External monitoring with per-region latency visibility helps you verify that the algorithm is actually distributing load as intended.