Real-time Updates

Polling, long polling, SSE, WebSockets and fanning updates out across servers.

Chat messages, live scores, ride locations, collaborative cursors, streaming LLM responses: many systems need to push updates to users as they happen. The problem has two halves:

Client ↔ server — how does the server push data to the client?
Server ↔ server — when an event happens on one server, how does it reach the server holding the right user's connection?

Part 1: getting updates to the client

[!INTERVIEW] Don't default to WebSockets If the client only needs to *receive* updates, SSE is simpler — it's plain HTTP, reconnects automatically and works through most proxies. If updates arrive every 30 seconds, polling is fine. Reach for WebSockets when traffic is genuinely bidirectional and frequent.

Operating persistent connections

WebSockets and SSE make servers stateful: a specific server holds a specific user's connection. That has consequences:

Load balancing — use L4 load balancers or L7 ones with WebSocket support; balance by least connections.
Deploys — draining a server drops its connections; clients must reconnect gracefully (with jittered backoff to avoid a reconnect storm).
Missed messages — on reconnect, clients send the last event ID they received and the server replays what they missed from durable storage.

Part 2: routing updates between servers

A message for user B arrives at server 1, but B is connected to server 7. Options:

Pub/Sub

Each connection server subscribes to channels for the users (or rooms) it holds. Publishers send events to the channel; the pub/sub system delivers them to whichever server subscribed.

[Chat Service] -> {Redis Pub/Sub / Kafka} -> [Connection Server 1]
{Redis Pub/Sub / Kafka} -> [Connection Server 7] -> Client B

Redis Pub/Sub — very low latency, simple, but no durability; fine when the database is the source of truth and clients can catch up on reconnect.
Kafka — durable and replayable, but partition-based rather than per-user channels; servers usually consume partitions and filter.

Connection registry / consistent hashing

Keep a registry (e.g. in Redis) mapping userId → serverId, and send events directly to the right server. Or use consistent hashing so that a user (or a document) always connects to a predictable server — useful for collaborative editing, where all editors of a document should share a server.

Choosing a design

Chat app — WebSockets + pub/sub between servers + durable message store for offline delivery.
Live comments on a stream with millions of viewers — SSE to clients; partition viewers by stream across servers; fan out via pub/sub; sample or batch comments for very hot streams.
Ride tracking — driver apps send locations via HTTP or WebSocket; rider receives updates via SSE/WebSocket from a server subscribed to that ride.
Collaborative docs — WebSockets; consistent hashing on document ID so one server serializes edits.

[!TIP] Mention the failure modes What happens when a connection server dies? Clients reconnect to another server, re-register, and fetch missed events by last-seen ID. That one sentence shows you've thought past the happy path.