Fanning Out to Followers

Fan-out on write vs read, the celebrity problem, feed storage, ranking and deletes.

When one action needs to reach many people — a post appearing in every follower's feed, a message delivered to every member of a group, a notification to everyone watching an item — you have a fan-out problem. The central decision is when you do the fan-out work: when the content is written, or when someone reads. Design a News Feed walks through a full design; this pattern covers the reusable ideas.

Fan-out on read (pull)

Store each post once. When a user opens their feed, look up who they follow, fetch recent posts from each, merge by time, and return the top N.

Writes are cheap: one insert per post.
Reads are expensive: a user following 500 accounts triggers up to 500 lookups and a merge, on every feed load.
Always fresh, and nothing is wasted on inactive users.

Fan-out on write (push)

When someone posts, immediately insert the post ID into a precomputed feed for each follower — typically a capped list in Redis.

Reads are trivial: one lookup of a ready-made list.
Writes multiply: a post by someone with 2,000 followers is 2,000 writes. Done asynchronously, from a queue.
Wasted work for followers who never log in, and a short delay before the post shows up everywhere.

[Post Service] -> (Posts DB)
[Post Service] -> {Fan-out queue} -> [Fan-out Workers] -> (Feed cache: post IDs per user)
[Feed Service] -> (Feed cache: post IDs per user)
[Feed Service] -> (Posts DB)

The feed cache stores post IDs, not posts. The feed service reads the IDs, then fetches (and caches) the post bodies — so an edit or delete only touches one copy of the post.

The celebrity problem

Fan-out on write breaks for accounts with millions of followers: one post becomes 50M feed writes, taking minutes and hammering the cache. The standard answer is a hybrid:

Normal accounts: fan-out on write.
High-follower accounts (above a threshold, say 100k–1M followers): don't fan out. Store their posts once.
At read time: take the user's precomputed feed, add recent posts from the few celebrities they follow (fetched and cached once per celebrity, shared by all readers), merge and rank.

precomputed feed (pushed):   [p981, p975, p960, p944, …]
celebrity posts (pulled):    [p979 from @popstar, p950 from @athlete]
merged by time / score:      [p981, p979, p975, p960, p950, p944, …]

You can apply the same idea to readers: skip pushing to users who haven't been active for 30 days, and rebuild their feed on read when they come back.

Storing feeds

Cap each feed at a few hundred to a thousand IDs. Hardly anyone scrolls further; older items come from a slower pull path.
Size it: 200M active users × 800 IDs × 8 bytes ≈ 1.3 TB of IDs — a sharded Redis cluster, sharded by user ID.
Use time-sortable IDs (see Generating Unique IDs) so merging and pagination work on IDs alone.
Paginate with a cursor (before_id = p944), not page numbers — new items arriving at the top would shift every page.

Ranking

Chronological feeds merge by ID. Ranked feeds score candidates on engagement, relationship strength and recency, usually in two stages:

Candidate generation — the precomputed feed plus pulled sources, a few hundred items.
Ranking — a model scores the candidates, and the top ones are returned (and the ranked page cached briefly so scrolling stays consistent).

Deletes, edits, unfollows and privacy

Delete or edit a post: change the single stored post; feed entries pointing to it are filtered out (or render the new version) on read. No need to chase millions of copies.
Unfollow: filter that author's posts at read time; clean the cached feed lazily.
Blocked or private: always check visibility at read time, never only at fan-out time — permissions can change after the push.

Fan-out beyond feeds

The same push-vs-pull choice applies to:

Group chat: small groups push each message to every member's inbox; very large channels are read from a shared channel log instead (see Design WhatsApp).
Notifications: push to a per-user notification list, with batching ("12 people liked your post") to avoid write storms.
Live updates: pushing over open connections to online users — see Real-time Updates and Broadcasting with Pub/Sub.

[!INTERVIEW] What a strong answer sounds like "For most users I'll fan out on write: a post goes to Kafka, and fan-out workers append its ID to each follower's feed list in Redis, capped at 800. Accounts over 500k followers are skipped; their recent posts are cached once and merged into the feed at read time. Feeds store IDs only, so deletes and edits touch one place, and visibility is rechecked when we hydrate. Inactive users aren't pushed to — we rebuild their feed on their next visit."