Load Balancing

The unsung hero of scalability. It protects your servers from crushing load and gives you the freedom to fail without downtime.

What you will learn

• Differentiate Layer 4 (Transport) vs Layer 7 (Application) Load Balancing
• Visualize the Algorithms: Round Robin, Least Connections, Hashing
• Understand the 'Sticky Session' Trap
• Master Health Checks: The heartbeat of the system

Load Balancing

Why This Matters In Interviews

Load balancing is where "just add more servers" becomes a real architecture answer instead of a slogan. Interviewers use this topic to test whether you understand how traffic is distributed, how failures are isolated, and why horizontal scaling needs coordination at the entry point.

Mental Model

A load balancer is the public traffic layer that turns a set of backend servers into one usable service. Clients should not need to know which server is healthy, overloaded, or newly added. The balancer owns that decision.

Core Tradeoff

The main tradeoff is simplicity versus smarter routing. Basic algorithms like round robin are cheap and predictable. Smarter strategies like least connections or Layer 7 routing adapt better to uneven traffic but require more state, deeper inspection, and more operational logic.

Common Mistake

WARNING: Engineers often think load balancing alone solves scalability. It only distributes traffic. If your application tier is stateful, your database is the bottleneck, or health checks are weak, the balancer just spreads failure more evenly.

Practice Next

Open the Load Balancer Simulator to compare algorithms under different traffic patterns. Then continue with API Gateway to see what happens when the edge must do more than simple distribution.

You have built a website. It lives on a server with an IP address 1.2.3.4. Suddenly, you go viral. 100,000 users hit 1.2.3.4 at once. The CPU melts. The server dies. The users leave.

The solution is not just "Add more servers". If you add a second server at 5.6.7.8, how does the user know to go there instead of the dead one?

You need a Traffic Cop. You need a Load Balancer.

1. The Concept

A Load Balancer (LB) sits between the User and your Server Farm. The user only knows the LB's IP. The LB knows everyone's IP.

1. User sends request to LB.
2. LB chooses a healthy server (e.g., Server A).
3. Server A processes it and replies to LB.
4. LB replies to User.

Crucial Role: The LB is the "Public Face" of your system. It shields your internal architecture from the world.

2. Layer 4 vs Layer 7: The Analogy

Engineers argue about L4 vs L7 constantly. Here is the difference.

Layer 4 (Transport Layer) - "The Traffic Cop"

The L4 LB sees packets. It sees Source IP and Destination Port. It does not look inside the packet.

• Analogy: You are a Traffic Cop directing cars. You see a Red Ferrari. You wave it to Lane 1.
• You don't know who is driving.
• You don't know if they are delivering pizza or diamonds.
• Pros: Ultra fast. Millions of requests/sec.
• Cons: "Dumb". Can't make decisions based on content.

Layer 7 (Application Layer) - "The Hotel Receptionist"

The L7 LB sees Data. It decrypts the HTTPS, reads the URL, the Headers, and the Cookies.

• Analogy: You are a Receptionist. You stop the guest. You ask to see their ID.
• You see they are VIP. -> "Go to the Penthouse (Service A)."
• You see they are Staff. -> "Go to the Basement (Service B)."
• Pros: Smart. Can route /api/video to powerful servers and /api/text to cheap ones.
• Cons: Slower. Requires CPU to decrypt/encrypt data (SSL Termination).

3. Algorithms: How to Choose a Victim

The LB has 10 servers. A request comes in. Who gets it?

Round Robin (The Fair Play)

"Server 1, then Server 2, then Server 3, then back to 1."

• Best for: Servers with identical specs and requests that take equal time.

Weighted Round Robin (The Inheritance)

"Server 1 is a Supercomputer (Weight 10). Server 2 is a Laptop (Weight 1)."

• The LB sends 10 requests to Server 1 for every 1 request to Server 2.

Least Connections (The Pragmatist)

"Server 1 has 100 active users. Server 2 has 5 active users. Send to Server 2."

• Best for: Weird workloads. If one user uploads a 1GB video (takes 10 minutes) and another fetches a logo (takes 10ms), Round Robin would kill the video server. Least Connections saves it.

IP Hash (The Stalker)

"User IP 1.2.3.4 always hashes to Server 5."

• Best for: Caching local efficiency.

4. The "Sticky Session" Trap

Beginners fall into this trap often. You store the User's Session (e.g., "Login = True") in the memory of Server A.

Imagine the user refreshes the page. The LB (Round Robin) sends them to Server B. Server B checks its memory. "I don't know who you are. Please Login." User Rage.

The Bad Fix: Sticky Sessions

You tell the LB: "If a user comes from IP X, always send them to Server A."

• Failure Mode: Server A crashes. All those users are now logged out and angry.

The Real Fix: Stateless Servers

Store the session in a shared database (like Redis).

• User hits Server A. Server A asks Redis "Is he logged in?" -> Yes.
• User hits Server B. Server B asks Redis "Is he logged in?" -> Yes.
• Result: You can destroy any server at any time, and users barely notice.

5. Health Checks: The Heartbeat

How does the LB know Server 3 is on fire? Health Checks.

Every 10 seconds, the LB sends a fake "Ping" to every server: GET /health.

• Server 1: "200 OK" -> Add to rotation.
• Server 2: "200 OK" -> Add to rotation.
• Server 3: "500 ERROR" (or Timeout) -> Remove from rotation immediately.

Self-Healing: This is how modern clouds survive. AWS Auto Scaling Groups notice the 500 Error, shoot Server 3 in the head, and spawn a fresh Server 4 to replace it.

6. Summary

The Architect's Decision Matrix

1. Microservices? Use L7. You need to route /video to the Video Service and /billing to the Billing Service.
2. SQL Database? Use L4. You just need to forward the TCP packets to the Read Replica.
3. State? Avoid Sticky Sessions. Use Redis.