Why the Algorithm Choice Actually Matters
A load balancer is only as smart as the algorithm behind it. Pick the wrong one and you'll find slow backends drowning in requests while fast ones sit idle, or stateful sessions randomly breaking mid-flow. This post compares three algorithms you'll encounter constantly: round-robin, least-connections, and hashing.
Round-Robin
Round-robin distributes requests sequentially across the backend pool. Request 1 goes to server A, request 2 to server B, request 3 to server C, then back to A.
When it works well
- Stateless services where requests have roughly equal cost (static file servers, simple REST APIs)
- Homogeneous backends—same CPU, memory, and network capacity
- Short-lived connections where distribution evens out quickly
Where it breaks down
Round-robin is blind to backend load. If one request spawns a 10-second database query and the next is a 10ms health check, they get the same weight. On heterogeneous fleets—common in cloud environments with mixed instance types—some nodes get crushed while others coast.
Weighted round-robin is a practical improvement. You assign a weight to each backend proportional to its capacity. Most load balancers (nginx, HAProxy, AWS ALB) support this out of the box. It still doesn't react to real-time load, but it's a reasonable static fix for mixed fleets.
Least-Connections
Least-connections routes each new request to the backend currently handling the fewest active connections. It's dynamic—the load balancer tracks connection counts in real time.
When it works well
- Long-lived connections: WebSockets, gRPC streams, database proxies
- Variable-cost workloads where some requests genuinely take much longer than others
- Backends that occasionally slow down due to GC pauses, hot caches, or noisy neighbors
Where it breaks down
Connection count is a proxy for load, not a measurement of it. A backend with 100 idle keep-alive connections looks "busier" than a backend doing 10 CPU-intensive operations. This is why some implementations use least-response-time instead, factoring in latency alongside connection count—HAProxy calls this leastconn with health checks, and Nginx Plus has least_time.
Also, in very high-throughput, short-connection scenarios the overhead of tracking and comparing connection counts can introduce unnecessary complexity without meaningful benefit over round-robin.
Hashing
Hashing routes requests based on a deterministic hash of some attribute—commonly the client IP, a request header, or a URL parameter. The same input always maps to the same backend.
When it works well
- Session persistence without sticky cookies (though cookies are often cleaner)
- Cache locality: if the same URL always hits the same backend, in-process caches stay warm
- Backends that maintain local state per user or resource
Consistent hashing
Simple modulo hashing (hash(key) % N) is fragile. Add or remove one backend and the majority of keys remap, spiking cache misses or breaking sessions.
Consistent hashing solves this. It arranges backends on a virtual ring so that adding or removing a node only remaps the keys that were previously owned by that node—typically around 1/N of all keys. This is the default in many distributed caches and is supported in HAProxy and nginx's hash directive with the consistent flag.
Where it breaks down
Hashing creates uneven distribution when the key space is skewed. If 30% of your traffic comes from one IP range—corporate proxies, for example—those backends carry a disproportionate load. Monitor actual backend traffic, not just connection counts.
Choosing the Right Algorithm: A Quick Decision Guide
- Are your requests stateless and roughly equal in cost? Start with round-robin. Add weights if your fleet is heterogeneous.
- Do you have long-lived or variable-cost connections? Use least-connections.
- Do you need cache affinity or session stickiness without cookies? Use consistent hashing.
- Are you running a mixed workload? Some load balancers let you layer strategies—e.g., consistent hashing for cache keys with least-connections as a fallback.
Monitoring Backs Up Your Algorithm Choice
None of these algorithms give you visibility on their own. You need external data to validate that distribution is actually working—backend error rates, response time percentiles per node, and whether any single backend is seeing a disproportionate share of failures.
This is where multi-region uptime monitoring earns its place. If one region's load balancer is misconfigured or a backend is silently returning 5xx errors only for certain hash keys, checks running from multiple geographic vantage points will surface that faster than internal metrics alone. Tools like Pingy run these checks continuously, so you catch regional failures rather than aggregating them away in a global average.
Key Takeaways
- Round-robin is simple and effective for homogeneous, stateless workloads; add weights for mixed fleets.
- Least-connections is better for long-lived or variable-cost connections, but connection count isn't the same as CPU load.
- Consistent hashing preserves cache locality and session affinity without remapping the entire key space on topology changes.
- Skewed key distributions will defeat hashing—monitor actual per-backend traffic.
- No algorithm replaces external monitoring; validate your distribution with real traffic data from multiple vantage points.