Why the Algorithm Choice Actually Matters
A load balancer is only as smart as its scheduling algorithm. Pick the wrong one and you end up with hot nodes drowning while others idle, sticky sessions breaking at inconvenient times, or cache hit rates cratering after a routine deployment. The algorithm isn't a set-and-forget detail — it's a core part of your traffic architecture.
Here's a clear-eyed look at the three most common options.
Round-Robin
Round-robin is the simplest algorithm: requests are distributed to each backend in sequence, cycling back to the start when the list is exhausted.
How it works
- Request arrives at the load balancer.
- Balancer selects the next server in the rotation.
- Pointer advances; next request goes to the following server.
Most implementations also support weighted round-robin, where higher-capacity nodes receive proportionally more requests.
When it works well
- Stateless services with homogeneous backends (same CPU, RAM, instance type)
- Short-lived, uniform requests — think simple REST API calls or static asset serving
- Situations where simplicity and predictability matter more than perfect distribution
Where it breaks down
Round-robin is oblivious to backend state. If one server is grinding through a slow database query while another is idle, round-robin will keep hammering both at the same rate. For workloads with high request-duration variance — long-polling, file uploads, complex queries — this causes real imbalance.
Least-Connections
Least-connections routes each new request to the backend with the fewest active connections at that moment.
How it works
- Request arrives.
- Balancer checks the current active connection count for each backend.
- Request is forwarded to the server with the lowest count. Ties are usually broken by round-robin fallback.
Variants include weighted least-connections (accounts for server capacity) and least response time (factors in latency alongside connection count, available in HAProxy and NGINX Plus).
When it works well
- Workloads with highly variable request duration (streaming, WebSockets, large uploads)
- Mixed-capacity backend pools where some nodes are faster or more powerful
- Microservice meshes where individual service response times fluctuate
Where it breaks down
Least-connections adds a small coordination overhead, which is negligible at most scales but worth noting in extremely high-throughput, short-duration scenarios. It also doesn't help if you need request affinity — the same client hitting the same server consistently.
Hashing (IP Hash / URL Hash / Consistent Hashing)
Hashing algorithms compute a hash of some attribute of the request — source IP, a header value, a URL path, or a session cookie — and map it deterministically to a backend.
How it works
- A hash function runs against the chosen key (e.g., client IP).
- The result maps to a specific backend, usually via modulo or a hash ring.
- The same key always routes to the same backend, as long as the pool is stable.
Consistent hashing (used by systems like Nginx's hash directive with consistent option, and most service meshes) minimizes remapping when backends are added or removed — only ~1/n keys remap instead of most of them.
When it works well
- Session-based applications that don't use a shared session store
- Cache-heavy backends where locality matters (the same request type always hits the same node's warm cache)
- gRPC or long-lived connection scenarios where you want connection reuse
Where it breaks down
Hashing is fragile when the pool changes. With basic IP hash, adding one server reshuffles a large portion of traffic, blowing out in-memory caches and breaking in-flight sessions. Consistent hashing mitigates this but doesn't eliminate it. It also creates imbalance if your key distribution is skewed — for example, if a significant portion of your traffic originates from a single NAT gateway IP.
Choosing the Right Algorithm
| Scenario | Recommended algorithm |
|---|---|
| Uniform, stateless API traffic | Round-robin |
| Variable request duration or mixed capacity | Least-connections |
| Session affinity or cache locality required | Consistent hashing |
| gRPC / WebSocket long-lived connections | Least-connections or hashing |
A Note on Monitoring
No matter which algorithm you choose, you need visibility into whether your load balancer is actually doing its job. Multi-region uptime monitoring (Pingy checks from multiple locations globally) helps you catch scenarios where a backend silently starts failing health checks and the load balancer starts concentrating traffic — something that looks fine from one vantage point but surfaces as latency spikes or errors from others.
Pair your algorithm choice with per-backend health checks and response time tracking. A misconfigured health check threshold is one of the most common reasons a theoretically sound load balancing setup degrades in production.
Key Takeaways
- Round-robin is simple and predictable but ignores backend state — use it for uniform, stateless workloads.
- Least-connections adapts to real load — better for variable request durations and mixed-capacity pools.
- Hashing provides affinity and cache locality but is sensitive to pool changes; prefer consistent hashing over simple modulo.
- Weighted variants of each algorithm exist and are worth enabling when your backends aren't homogeneous.
- The algorithm choice is inseparable from your health-check configuration — a backend that's up but degraded should be detected and down-weighted, not just routed to equally.