Why the Algorithm Matters
A load balancer is only as smart as the algorithm driving it. Pick the wrong one and you'll see hot spots, session breakage, or uneven resource exhaustion—even if your infrastructure is otherwise well-provisioned. This post walks through the three algorithms you'll encounter most often: round-robin, least-connections, and hashing.
Round-Robin
Round-robin distributes requests sequentially across the pool. Request 1 goes to server A, request 2 to server B, request 3 to server C, then back to A.
How it works
The balancer keeps a simple pointer to the next server in the list. There's no state to track beyond that pointer, which makes it extremely cheap to compute.
Weighted round-robin is a common extension: you assign a weight to each node (e.g., a node with 2× the CPU gets weight 2), and the balancer routes proportionally more traffic to it.
When to use it
- Requests are roughly uniform in cost (static assets, simple API calls)
- All backend nodes have similar specs
- You want the lowest overhead on the balancer itself
When it breaks down
Round-robin is blind to backend state. If one request takes 10 ms and the next takes 2 s, the balancer doesn't know or care—it keeps sending traffic at the same rate. This can cause one node to queue up work while others sit idle.
Least-Connections
Least-connections (also called least-outstanding-requests in some proxies) routes each new request to the node currently handling the fewest active connections.
How it works
The balancer maintains a counter per backend. When a connection opens, the counter increments; when it closes, it decrements. Incoming requests go to the node with the lowest count. Weighted least-connections factors in node capacity the same way weighted round-robin does.
When to use it
- Requests have highly variable processing time (database queries, file uploads, ML inference)
- Backend nodes have different throughput characteristics
- You're running long-lived connections (WebSockets, gRPC streams)
When it breaks down
Least-connections relies on accurate connection counts. With very short-lived requests, the counter fluctuates so fast that the algorithm can behave like random selection. It also adds a small amount of state that must be consistent—this gets complicated in distributed balancer setups without a shared data plane.
Hashing
Hashing routes requests based on a key derived from the request itself—most commonly the client IP, a session cookie, or a specific header. The same key always maps to the same backend node.
How it works
The balancer computes a hash of the key and maps it to a node, often using consistent hashing to minimize redistribution when nodes are added or removed. Consistent hashing arranges nodes on a virtual ring; a key routes to the first node clockwise from its hash position.
When to use it
- You need session affinity (sticky sessions) without storing session state in a shared cache
- Backends have local caches that benefit from cache locality (each node caches data for its slice of users)
- Stateful protocols that can't be interrupted mid-session
When it breaks down
If your key space is skewed—say, a large share of traffic comes from one IP range—you'll get uneven load. Consistent hashing mitigates redistribution on topology changes but doesn't solve hot-key problems. You also lose the ability to shift traffic away from a slow node without rehashing.
Choosing the Right Algorithm
Here's a quick decision guide:
- Homogeneous nodes + uniform requests → start with round-robin. It's simple, predictable, and easy to debug.
- Variable request cost or mixed node specs → use least-connections (or least-outstanding-requests if your proxy supports it—NGINX Plus, HAProxy, and Envoy all do).
- Session state or cache locality requirements → use IP hash or cookie-based consistent hashing. Prefer cookie-based if you control the client, since IP hashing breaks behind NAT or shared egress.
- Hybrid needs → some proxies let you combine strategies; for example, HAProxy supports
balance leastconnwithstick-tablefor session persistence on top.
Observability Is Not Optional
No algorithm protects you from a backend that has silently degraded. A node that's still accepting connections but responding in 8 s instead of 80 ms looks healthy to a round-robin balancer and only slightly penalized under least-connections.
This is where health checks and external monitoring close the loop. Active health checks (your balancer polling /health) catch hard failures. Passive checks (monitoring error rates and latency per upstream) catch soft degradation. Multi-region uptime monitoring—like what Pingy runs from multiple geographic vantage points—adds a layer that validates your load-balanced endpoint behaves correctly from the outside, not just from within your own network.
Key Takeaways
- Round-robin is the right default for uniform, stateless workloads; don't over-engineer when it's sufficient.
- Least-connections adapts to variable request cost and is the better default for anything with unpredictable latency.
- Hashing solves session affinity and cache locality but trades load evenness for key-based determinism.
- Weighted variants of all three algorithms let you account for heterogeneous node capacity.
- No algorithm compensates for missing observability—pair your balancer strategy with active health checks and external monitoring.