Load-balancing algorithms you can test with NGINX

Start with round-robin when any healthy backend can handle a request. Try least connections when requests overlap and their duration varies. Use hashing when sending the same key back to the same server has a measurable benefit, such as keeping a cache warm.

The choice depends on what your proxy counts and what happens when a backend disappears. This guide uses NGINX Open Source 1.28.3 for the configuration and lab results below.

Count the work your proxy routes

An HTTP request and a TCP connection are different units of work. A Layer 4 TCP load balancer selects a backend for a connection. An HTTP reverse proxy can select a backend for each request. Client keepalive therefore does not, by itself, promise that every request reaches the same application server.

Read the algorithm's definition for your proxy and protocol. For example, Envoy's least-request policy can choose between two randomly sampled hosts by comparing their active requests. That is different from counting TCP connections. With HTTP/2 multiplexing, one connection can carry several requests at once. See Envoy's load-balancing policies and HTTP/2 streams.

Round-robin gives each server a turn

With two equally weighted, available servers, a simple rotation gives each server roughly half the assignments. NGINX uses weighted round-robin by default; a weight of two gives a server twice the share of a server with weight one. NGINX's introductory guide explains the default and weighted behavior.

That share says nothing about CPU time. A server processing a slow report can still receive its next turn. Weights express a capacity assumption that you configure. They do not automatically measure spare memory or database congestion.

Use this as a baseline for a stateless application. Keep the request mix and traffic rate fixed when comparing another policy. If all requests finish before the next begins, a test tells you little about scheduling under load.

Least connections needs overlapping work

NGINX's least_conn selects the server with the fewest active connections, accounts for weights, and uses weighted round-robin when the counts tie. The upstream reference defines that behavior.

The lab below holds one request open on server A. While it is still active, twelve short requests all go to B. That is the behavior you should reproduce before relying on this policy for uploads or streams.

A connection count is only a proxy for work. An idle WebSocket and a request doing expensive computation can each occupy a connection. A server returning errors immediately can have a low active count too. Keep health checks and application metrics in the decision.

With multiple NGINX workers, use an upstream zone so they share connection and failure counters. Otherwise each worker has its own view. The NGINX administration guide describes this limitation. Sharing those counters on one proxy does not share them across separate proxy machines.

Hashing keeps a key near its previous backend

Hashing selects a server from a request key. For a cache, that key could identify the object being fetched. Consistent hashing reduces the number of keys that change servers when the pool changes. It does not guarantee an even traffic rate: one heavily requested key can still concentrate work on one server.

NGINX supports hash $uri consistent;. Here, $uri is the normalized path and excludes the query string. That choice deliberately sends /item/42?size=small and /item/42?size=large to the same server. If those are separate cache objects, decide whether sharing a backend helps your application. The hash directive documents the mapping, and the core variables reference defines the key.

Choose the key from the behavior you need. A client IP can combine many people behind a shared gateway. A missing session cookie can leave many requests with the same empty key. Test those cases as well as a uniform set of keys.

Affinity also has other implementations. HAProxy can insert a routing cookie and use it to remember a server assignment; its session-persistence guide also covers IP-based stick tables. Hashing is one option. Any affinity scheme still needs a plan for session data when the selected server fails. Routing a request elsewhere cannot recover state that existed only in that server's memory.

The supporting diagram below shows illustrative choices for equally weighted servers. Its active counts describe one scheduling instant.

A small NGINX configuration to test

This is a complete lab configuration for an isolated proxy. It expects the name backend to resolve to a test host serving HTTP on ports 8081 and 8082. Each endpoint should return its identity, a or b. Replace those addresses for your own test environment. The listener has no TLS or authentication; use a loopback-bound container port or a private lab network.

worker_processes 1;
events { worker_connections 1024; }
http {
    upstream app_pool {
        zone app_pool 64k;
        server backend:8081 max_fails=1 fail_timeout=5s;
        server backend:8082 max_fails=1 fail_timeout=5s;
    }
    server {
        listen 8080;
        location / {
            proxy_pass http://app_pool;
            proxy_connect_timeout 1s;
            proxy_read_timeout 15s;
            proxy_next_upstream error timeout http_503;
            proxy_next_upstream_tries 2;
        }
    }
}

Save this as a separate lab nginx.conf and validate that exact file:

nginx -t -c /absolute/path/to/lab/nginx.conf

For least connections, add least_conn; inside upstream app_pool, before the server lines. For consistent hashing, use hash $uri consistent; in that position instead. Test one policy at a time. For weighted round-robin, leave both directives out and add weight=2 to the first server line.

In our container lab, port 8080 was published only at 127.0.0.1:18089. These requests collect backend identities:

for i in 1 2 3 4 5 6 7 8 9 10 11 12; do
    curl -fsS http://127.0.0.1:18089/item/42
done

For the overlap test, the lab endpoint /hold waits eight seconds before responding. Start it in the background, confirm in the backend log that it has arrived, then run the twelve requests while it remains open. A normal endpoint that returns immediately will not reproduce that test.

What the lab showed

These are routing observations from NGINX 1.28.3 with one worker and synthetic HTTP backends. They are not throughput measurements. Each policy started with a fresh proxy process; the overlap case used equal weights.

The remapping counts depend on the keys and server identities. Repeat the measurement with your own distribution before estimating how much cache warming a pool change will require. Also test a popular key repeatedly; spreading distinct keys across servers does not spread repeated requests for one key.

Failure behavior is a separate choice

The example explicitly retries HTTP 503, connection errors and timeouts, with at most two upstream attempts. In this two-server pool, one qualifying failure marks a server unavailable for five seconds. This is passive failure handling driven by requests. It does not poll a readiness endpoint in the background. See NGINX's failure parameters.

Retry rules matter when a request changes data. NGINX normally avoids passing a non-idempotent request such as a POST to another server once it has been sent upstream. The non_idempotent option changes that protection. Retrying also becomes impossible after response data has gone to the client. Read the retry directive before adapting this GET-only test to writes.

After a backend returns, test session behavior and cache misses as well as successful HTTP status. Keep per-backend latency, error rate and application queue depth alongside external checks of the public endpoint. If a policy lowers average latency while a subset of users loses sessions during a restart, the test has found a correctness problem that the average cannot show.