Nginx can act as a load balancer in front of several application servers, and the upstream block decides which server receives each request. Open source Nginx ships six balancing methods: round robin, weighted round robin, least_conn, ip_hash, hash and random. In this tutorial you will build a small lab on one Ubuntu 24.04 server with three test backends, switch between each method, watch how traffic is distributed, and add passive health checks, a backup server and upstream keepalive.

Prerequisites

To follow this tutorial you need:

  • A server running Ubuntu 24.04 LTS (for example a CubePath VPS). The same steps work on Debian 12.
  • A non-root user with sudo privileges.
  • Nginx installed from the Ubuntu repository (sudo apt install nginx). Ubuntu 24.04 ships Nginx 1.24.

In production the backends are separate servers. Here they are extra Nginx server blocks on 127.0.0.1 so you can see the effect of each algorithm without building a cluster.

Step 1 - Creating three test backends

Create a configuration file for the backends. Each one listens on a local port and returns its own name, which makes it easy to see where a request landed:

sudo nano /etc/nginx/conf.d/backends.conf

server {
    listen 127.0.0.1:8081;
    default_type text/plain;
    return 200 "backend 1\n";
}

server {
    listen 127.0.0.1:8082;
    default_type text/plain;
    return 200 "backend 2\n";
}

server {
    listen 127.0.0.1:8083;
    default_type text/plain;
    return 200 "backend 3\n";
}

Test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Check that each backend answers:

curl -s http://127.0.0.1:8081 http://127.0.0.1:8082 http://127.0.0.1:8083

backend 1
backend 2
backend 3

Step 2 - Configuring the load balancer with round robin

Disable the default site so the load balancer can own port 80:

sudo rm /etc/nginx/sites-enabled/default

Now create the load balancer. The upstream block lists the pool of servers, and proxy_pass sends requests to it. With no method specified, Nginx uses round robin: each request goes to the next server in the list.

sudo nano /etc/nginx/conf.d/lb.conf

upstream app_backend {
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;

    location / {
        proxy_pass http://app_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Reload and send six requests:

sudo nginx -t && sudo systemctl reload nginx
for i in $(seq 6); do curl -s http://localhost/; done

backend 1
backend 2
backend 3
backend 1
backend 2
backend 3

Round robin is the right default when backends are identical and requests cost roughly the same.

In the following steps you only edit the upstream block in /etc/nginx/conf.d/lb.conf. After each change, run sudo nginx -t && sudo systemctl reload nginx and repeat the curl loop.

Step 3 - Weighting servers with different capacity

If one backend has more CPU or memory than the others, give it a higher weight. The default weight is 1.

upstream app_backend {
    server 127.0.0.1:8081 weight=3;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}

Send ten requests and count the answers:

for i in $(seq 10); do curl -s http://localhost/; done | sort | uniq -c

      6 backend 1
      2 backend 2
      2 backend 3

Backend 1 now receives three out of every five requests. Weights also apply to least_conn, hash and random.

Step 4 - Sending traffic to the least busy server with least_conn

least_conn sends each request to the server with the fewest active connections, taking weights into account. It helps when request duration varies a lot, for example an API where some calls return in 10 ms and others run reports for 20 seconds. Round robin would keep piling requests on a server that is still busy with slow ones.

upstream app_backend {
    least_conn;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}

With the instant test backends every connection finishes immediately, so the output looks like round robin. The difference only shows under concurrent, long-running requests.

Step 5 - Keeping clients on the same server with ip_hash

Applications that keep sessions in local memory or on local disk need each client to return to the same backend. ip_hash uses the first three octets of the client IPv4 address (or the whole IPv6 address) as the key, so a client keeps hitting the same server as long as it is up.

upstream app_backend {
    ip_hash;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}

All test requests come from 127.0.0.1, so they all land on one backend:

for i in $(seq 6); do curl -s http://localhost/; done | sort | uniq -c

      6 backend 2

To take a server out of an ip_hash pool without remapping every other client, mark it down instead of deleting the line:

    server 127.0.0.1:8083 down;

Keep two limitations in mind:

  • Many users behind the same NAT or corporate proxy (same /24) all go to one server, which can unbalance the pool.
  • If Nginx sits behind another proxy or a CDN, every request comes from the proxy address. Use the realip module or the hash method on a header instead.

Storing sessions in Redis or a database removes the need for stickiness and is the more robust fix.

Step 6 - Hashing on any key with hash

The hash method lets you choose the key. Hashing on the request URI sends the same URL to the same backend every time, which improves cache hit rates on caching backends. The consistent parameter enables ketama consistent hashing, so adding or removing a server only remaps a small share of keys.

upstream app_backend {
    hash $request_uri consistent;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}

Request three different paths twice each:

for p in a b c a b c; do printf "/%s -> " "$p"; curl -s "http://localhost/$p"; done

/a -> backend 3
/b -> backend 1
/c -> backend 3
/a -> backend 3
/b -> backend 1
/c -> backend 3

The exact mapping on your server will differ, but a given path always returns the same backend. Other useful keys are $cookie_sessionid for cookie based stickiness and $http_x_forwarded_for when a trusted proxy sits in front.

Step 7 - Choosing randomly with random two

random picks a server at random, respecting weights. With two, Nginx picks two servers at random and then sends the request to the one with fewer active connections. This "power of two choices" approach works well when several load balancers share the same backends, because they do not need to agree on state.

upstream app_backend {
    random two least_conn;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
}

for i in $(seq 30); do curl -s http://localhost/; done | sort | uniq -c

     11 backend 1
      9 backend 2
     10 backend 3

Step 8 - Adding passive health checks and a backup server

Open source Nginx detects failed servers passively. When a connection to a backend fails, Nginx retries the request on the next server (proxy_next_upstream) and counts the failure. After max_fails failures within fail_timeout, the server is skipped for fail_timeout seconds, then tried again.

Go back to round robin and simulate an outage by pointing the third server at port 8084, where nothing listens. Also add a backup server that only receives traffic when all primary servers are unavailable:

upstream app_backend {
    server 127.0.0.1:8081 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8082 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8084 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8083 backup;
}

Inside the location / block, define which errors trigger a retry and limit the retries:

        proxy_next_upstream error timeout http_502 http_503;
        proxy_next_upstream_tries 2;
        proxy_connect_timeout 2s;

Reload and run the test loop:

sudo nginx -t && sudo systemctl reload nginx
for i in $(seq 6); do curl -s -o /dev/null -w "%{http_code}\n" http://localhost/; done

200
200
200
200
200
200

Clients never see an error. The failed attempts are recorded in the error log:

sudo tail -n 3 /var/log/nginx/error.log

2026/09/25 10:12:04 [error] 2211#2211: *31 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8084/", host: "localhost"

The backup parameter cannot be combined with hash, ip_hash or random. Only retry non-idempotent requests (POST, PATCH) with care: by default Nginx does not pass them to the next server once they were sent, and you should keep it that way unless your application is safe against duplicates.

Change 8084 back to 8082 or 8083 when you are done testing.

Step 9 - Reusing upstream connections with keepalive

By default Nginx opens a new TCP connection to the backend for every request. keepalive keeps a number of idle connections per worker open for reuse, which lowers latency and avoids exhausting ephemeral ports under load. It needs HTTP/1.1 and an empty Connection header towards the backend:

upstream app_backend {
    least_conn;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
    server 127.0.0.1:8083;
    keepalive 32;
}

In location /, add:

        proxy_http_version 1.1;
        proxy_set_header Connection "";

Place keepalive after the balancing method line. Reload and confirm that requests still succeed:

sudo nginx -t && sudo systemctl reload nginx
curl -s http://localhost/

Choosing the right algorithm

MethodPicks the server byUse it when
Round robin (default)Next in the listIdentical servers, similar request cost
weight=Round robin with proportionsServers of different sizes
least_connFewest active connectionsRequest duration varies widely, long-lived connections
ip_hashClient IP addressSession stickiness without cookies, clients not behind a shared proxy
hash key consistentAny variableCache affinity by URI, stickiness by cookie or header
random two least_connBest of two random picksSeveral load balancers sharing one pool

Troubleshooting

  • nginx: [emerg] unknown directive "least_time" or "health_check": these are NGINX Plus features. Use least_conn and passive checks with max_fails.
  • nginx -t fails with an error about the backup parameter: you combined backup with hash, ip_hash or random. Remove one of them.
  • All traffic goes to one backend with ip_hash: every request comes from the same address, usually a proxy or CDN in front of Nginx. Configure set_real_ip_from and real_ip_header, or hash on a header or cookie.
  • 502 Bad Gateway for every request: all servers in the pool failed. Check sudo tail -f /var/log/nginx/error.log and test each backend directly with curl.

Conclusion

You configured every load balancing method available in open source Nginx, measured how each one distributes requests, and added passive failover, a backup server and upstream keepalive. Start with round robin or least_conn, and only move to hash based methods when you need affinity.

As next steps, replace the test backends with your real application servers, add TLS termination with Let's Encrypt in front of the upstream, and consider HAProxy if you need active health checks without NGINX Plus.