How a load balancer spreads requests

· 2 min read

#how-it-works#networking

A load balancer receives requests at one address and passes each one to one of several servers that do the same job. The idea to keep: no single server has to carry everything, and a server that fails is skipped.

The short version

Clients talk to the load balancer as if it were the server. Behind it sit a few servers. For every request the load balancer picks one by a rule, and when a server keeps failing it stops sending requests there for a while. A CDN keeps copies of files near the visitor, while a load balancer shares the work behind one address; the first idea is in How a CDN serves files from a server near you.

How a load balancer spreads requests A client sends a request to a load balancer. The load balancer picks one of three servers for each request. Server A and Server B receive requests. Server C has failed, so a dashed arrow shows that it is skipped. Client Load balancer Server A Server B Server C failed, skipped 1 request 2 pick one
Simplified: the load balancer picks a healthy server for each request and skips the one that failed.

Step by step

  1. A client sends a request to the load balancer's address.
  2. The load balancer picks a server with its method, shown in the table.
  3. The server answers, and the reply goes back to the client.
  4. A server that keeps failing is left out; in nginx's defaults, one failed attempt sidelines it for 10 seconds.
  5. Once it responds again, it takes part again.
Method How it picks the server
Round robin The next one in turn, weighted if you set weights; the nginx default
Least connections The one with the fewest active connections; useful when some requests run long
IP hash Derived from the client's IP address, so the same client usually reaches the same server

Why it matters

Spreading requests lets several servers share the work, and one failing server need not stop the service. With IP hash the same client keeps landing on one server, and if that server is unavailable the client moves to another.

Sources