Spaced Repetition (Lesson 2.4): Why is SSE one-directional (server→client only)? And for a notification use case like TaskFlow's, why might SSE often still be a better choice than WebSocket despite that limitation?
Prerequisite: Lesson 3.1 (Load Balancer, L4/L7)
By the end of this lesson you will be able to:
- Understand how Round Robin, Weighted Round Robin, and Least Connections work, and when each fits.
- Grasp the basic idea of IP hash and consistent hashing (in the context of session affinity) — an introduction, with full depth in Module 10.1.
- Look at TaskFlow's specific traffic pattern and pick the right algorithm.
Tier: 3 — Design Exercise
0. Where TaskFlow Is Right Now
In Lesson 3.1 we learned that a load balancer decides where a request goes (L4 vs L7, which backend pool). But one question is still open — if three identical servers sit behind the LB, which one does this particular request go to right now? At random? In turn? By some other logic? That is today's subject — the decision algorithm inside the LB.
1. Theory
1.1 Round Robin — the Simplest, Taking Turns
Round Robin is the simplest algorithm — requests go to each server in turn, in order:
Request 1 -> Server A
Request 2 -> Server B
Request 3 -> Server C
Request 4 -> Server A (back to the start)
Request 5 -> Server B
...
Advantage: the easiest to implement, with no extra state to track (a single "who's next" counter is enough).
Problem: it assumes every server has equal capacity and every request costs roughly the same to process. When either assumption is false, problems appear. For example, if Server A's hardware is weaker than B's and C's, Round Robin still gives all three an equal share, and Server A overloads quickly.
Weighted Round Robin is one fix — each server gets a "weight" matching its capacity:
Server A (weight 1) — weaker hardware
Server B (weight 2) — stronger hardware
Server C (weight 2) — stronger hardware
Distribution: A, B, C, B, C, A, B, C, B, C ...
(B and C receive twice as many requests as A)
1.2 Least Connections — Sending to Whoever Is Least Busy
Round Robin has a deeper problem — it knows "who gets the next one" but not how busy each server is right now. Take TaskFlow's "Export to PDF" feature (remember the Module 1 exit challenge?) — those requests can take seconds to process, while an ordinary GET /api/tasks finishes in milliseconds. If Round Robin keeps handing Server A these heavy PDF export requests, it will keep sending it new ones even while it is busy, simply because "its turn has come" — which is not fair and creates genuinely unequal load.
The Least Connections algorithm fixes this — it tracks how many active connections or requests each server currently has, and always sends the new request to the least busy one:
Right now:
Server A — 12 active connections (processing heavy requests)
Server B — 3 active connections
Server C — 5 active connections
A new request arrives -> it goes to Server B (least busy)
When this matters most: when request processing times vary widely (some fast, some slow) — then splitting request counts evenly (Round Robin) is not enough, and looking at actual load (who is busy right now) is far more accurate.
1.3 IP Hash / Session Affinity — Same Client, Same Server (an Introduction)
Sometimes you want the same client to keep reaching the same server (called session affinity or a sticky session). One simple implementation hashes the client's IP address and picks a server from that hash:
hash(client_IP) % number_of_servers = which server
Why you might need this: recall the stateful vs stateless discussion from Lesson 1.6 — suppose TaskFlow rushed out a feature where session data still lives in local server memory (not yet migrated to Redis; not ideal, but this kind of technical debt is real). In that situation, sending the same user to the same server every time (via IP hash) can be a temporary workaround until it is properly made stateless.
But there is a big problem — adding or removing a server upends everything:
With 3 servers: hash(IP) % 3
Add one to make 4: hash(IP) % 4
The same client IP can give completely different results under % 3 and % 4 —
meaning nearly every client suddenly lands on a different server!
That is exactly the problem consistent hashing solves — a more sophisticated hashing technique where adding or removing a server re-maps only a small fraction of clients, not almost all of them. It is such an important and deep topic (used not only in load balancing but in distributed database sharding, CDN routing, and much else) that we have given it an entire separate lesson — Module 10.1. For today, just know this: the problem (all hashes collapsing when the server count changes) exists, and there is an elegant solution we will go into later.
Trade-off Table — LB Algorithms
| Algorithm | What it considers | Best fit | Limitation |
|---|---|---|---|
| Round Robin | Nothing, just order | Uniform server capacity, uniform request cost | Unfair when capacity or request cost varies |
| Weighted Round Robin | Server capacity (manual weight) | A pool of servers with different capacities | Weights must be set manually; not dynamic |
| Least Connections | Current active load | Widely varying request processing times | Slightly more overhead (state must be tracked) |
| IP Hash | Client identity | When session affinity is needed (a workaround) | Big disruption when servers are added or removed (without consistent hashing) |
2. Interview Angle
A common question — "some of your backend endpoints are fast (a few ms) and some are slow (several seconds) — which LB algorithm would you choose?" The right answer is Least Connections, because under Round Robin the slow requests can tie up a server while its next "turn" brings it more new requests anyway, breaking fair load distribution.
Another question that connects directly to Lesson 1.6 — "is session affinity (sticky sessions) good practice?" A good answer says it is a workaround, not the ideal solution. The right approach is making the application stateless (Lesson 1.6, an external session store), so that any request can go to any server with no session affinity at all — which makes horizontal scaling maximally flexible and resilient (if one server goes down, no user is blocked, because nothing was pinned to it).
3. Key Takeaway
- Round Robin — simple, sequential, assuming uniform capacity and cost
- Weighted Round Robin — handles differing server capacity with manual weights
- Least Connections — routes by current active load, fairer when request costs vary
- IP hash / session affinity — pins a client to one server, but changing the server count causes big disruption
- Consistent hashing solves that disruption problem — details in Module 10.1
- Session affinity is a workaround; a stateless architecture (Lesson 1.6) is the ideal long-term solution
4. New Terms (Glossary)
| Term | Meaning |
|---|---|
| Round Robin | An algorithm sending requests to each server in turn, in order |
| Weighted Round Robin | Splitting requests in different proportions according to each server's capacity |
| Least Connections | Sending a new request to whichever server has the fewest active connections |
| Session Affinity (Sticky Session) | Repeatedly sending the same client to the same backend server |
| IP Hash | Picking a specific server by hashing the client's IP |
| Consistent Hashing | A hashing technique where most mappings survive a change in the server count (details later) |
5. Reflection Questions
- Most TaskFlow endpoints (task CRUD) are fast and cost roughly the same, but "Export to PDF" (remember the exit challenge?) arrives occasionally and takes time. Which algorithm would you propose for this mixed traffic pattern, and why?
- If all of TaskFlow's servers were properly stateless (per Lesson 1.6), would there be any need for IP hash or session affinity? Why or why not?
Answer Key
Question 1: Least Connections is the most sensible here, because request processing costs vary widely (fast CRUD vs slow PDF export). Under Round Robin, a server busy with a PDF export still receives new requests on its next turn, overloading it further. Least Connections automatically avoids the busy server and sends new requests to a less busy one.
Question 2: If every server is genuinely stateless (sessions, files, everything in external stores), then IP hash and session affinity are unnecessary — any server can handle any request without depending on where the previous one went. In that case Round Robin or Least Connections is better, because they distribute traffic most flexibly and evenly, with no unnecessary constraint like session affinity.
6. Practical Exercise
Tier 3 — Design Exercise
Scenario: TaskFlow has four server instances. Which LB algorithm would you propose for each of the three situations below, and why:
- All four servers have identical hardware, and every TaskFlow endpoint is roughly equally fast (an ordinary CRUD app)
- One server was recently upgraded (double the CPU and RAM), while the other three are still on the old hardware
- TaskFlow has a legacy feature still keeping some temporary state in local server memory (making it stateless is still pending, marked as technical debt) — for now you have to work around this limitation
7. Progress Ledger
=== PROGRESS LEDGER ===
Completed: Module 1 (complete) + Module 2 (complete), 3.1
Current: 3.2 — LB Algorithms
TaskFlow state: multi-instance transition — stateless Express server instances,
a single Postgres, a ~100-user base, with the horizontal-scale architecture now established
Terms learned (Module 3 so far): Load Balancer, L4/L7, SSL Termination,
Content-based Routing, Round Robin, Weighted Round Robin, Least Connections,
Session Affinity, IP Hash, Consistent Hashing (intro)
Weak spots: reaching the right answer but justifying it with the wrong or irrelevant reason
(the Idempotency-Key vs Host-header confusion in 3.1) — build the habit of identifying
exactly which specific factor drives a decision
Next: 3.3 — Reverse Proxy vs Forward Proxy, Nginx Hands-on
=======================
8. Next Lesson
Send the exercise over. When you are ready, write next — we move to Lesson 3.3: reverse proxy vs forward proxy, where we do our first Nginx hands-on (Tier 2 — a Docker and config-based exercise).