How a load balancer spreads requests
A load balancer receives requests at one address and passes each one to one of several servers that do the same job. The idea to keep: no single server has to carry everything, and a server that fails is skipped.
The short version
Clients talk to the load balancer as if it were the server. Behind it sit a few servers. For every request the load balancer picks one by a rule, and when a server keeps failing it stops sending requests there for a while. A CDN keeps copies of files near the visitor, while a load balancer shares the work behind one address; the first idea is in How a CDN serves files from a server near you.
Step by step
- A client sends a request to the load balancer's address.
- The load balancer picks a server with its method, shown in the table.
- The server answers, and the reply goes back to the client.
- A server that keeps failing is left out; in nginx's defaults, one failed attempt sidelines it for 10 seconds.
- Once it responds again, it takes part again.
| Method | How it picks the server |
|---|---|
| Round robin | The next one in turn, weighted if you set weights; the nginx default |
| Least connections | The one with the fewest active connections; useful when some requests run long |
| IP hash | Derived from the client's IP address, so the same client usually reaches the same server |
Why it matters
Spreading requests lets several servers share the work, and one failing server need not stop the service. With IP hash the same client keeps landing on one server, and if that server is unavailable the client moves to another.
Sources
- Module ngx_http_upstream_module — nginx — round robin as the default,
least_conn,ip_hash, and themax_failsandfail_timeoutdefaults - Using nginx as HTTP load balancer — nginx — the methods, least connections for long requests, IP hash for session persistence, and how a failed server is probed again
- How Elastic Load Balancing works — AWS — the load balancer monitors target health and routes only to healthy targets