Design a Social Media News Feed

Advanced System Design & Software Architecture · Lesson 57 · Real-World System Design Case Studies

Lesson Overview

An instructor with four million followers posts that the exam moved to Thursday. A learner who follows eighty classmates posts a study question. If TechLearningPro's community feed copies both posts into every follower's timeline at write time, the exam announcement tries to write four million timeline entries before anyone can read it, and the study question is cheap. If the feed instead builds every timeline at read time by querying every account the viewer follows, the eighty-classmate case becomes eighty queries on the hot path and the exam announcement is a single author lookup. Neither pure strategy survives the shape of this graph.

The home feed is a derived view. The system of record is posts plus the follow graph, not the timeline cache. Fan-out on write pushes a post id into the timelines of followers who are cheap to update. Fan-out on read leaves a post on the author's timeline and merges it when someone opens the feed. The celebrity problem is the long tail of follower counts: the average learner follows dozens of people and is followed by dozens, while a handful of instructors are followed by millions. Designing for the average write amplification hides the spike that takes the cluster down. This lesson uses a hybrid. Accounts under a follower threshold are fanned out on write into a bounded per-user cache. Accounts over the threshold are merged at read from a short per-author list. The viewer's feed dedupes by post id so a threshold crossing cannot show the exam post twice.

Ranking is a second stage, not a storage layout. The candidate set is chronological. A ranker may reorder the first page when it is healthy and must be skippable when it is not, because a learner who follows an instructor needs the Thursday announcement even if the model is down. Pagination is a cursor. Offset pagination skips and repeats rows as new posts insert at the top of a hot feed.

Media bytes live in the object store from Design a Cloud File Storage System. This lesson stores media references, not video pipelines. Chat is Design a Messaging System. Notifications that "someone you follow posted" are Design a Notification Platform and are a different fan-out, with preferences and receipts. Do not implement the feed by sending each follower a push.

Module 6 will ask you to defend the threshold, the rebuild path when the cache is cold, and the authorization check that a shared post-body cache cannot replace. System Design Interview Framework uses this hybrid as the canonical "it depends" answer, provided you compute the amplification instead of reciting the slogan.

Learning Objectives

By the end of this lesson, you should be able to:

1. Separate the system of record (posts, follows, blocks) from derived timelines.
2. Compute write amplification for an average user and for a celebrity, and show why the average is the wrong design input.
3. Specify a hybrid fan-out with an explicit follower threshold, including what happens when an account crosses it.
4. Describe read-path merge, dedupe, and a bounded celebrity set per viewer.
5. Choose a cursor pagination scheme and explain the failure of large offsets.
6. Place a ranker behind a timeout with a chronological fallback.
7. Define cache keys, caps, and rebuild so a Redis loss degrades latency and does not delete posts.
8. Enforce visibility, blocks, and mutes at fan-out and again at read.
9. Estimate feed reads per second and timeline memory for a learning-community workload.
10. Compare keeping full post bodies in the timeline versus storing ids and hydrating bodies once.
11. Review freshness, thundering-herd rebuild, and private-post leakage.
12. Defend the design in a Module 6 interview with numbers, not with "we'll use a cache."

Prerequisites

• Lesson 25 and lesson 27: read amplification, cache-aside, and why a timeline is a cache with a rebuild function.
• Lesson 28 and lesson 30: indexes on (author_id, created_at) and on (followee_id) so fan-out and profile timelines stay index-only.
• Lesson 33: the post-created event is at least once. Timeline inserts are idempotent on (user_id, post_id).
• Lesson 35: post rate limits per author so a bug or a bot does not fan out a million writes.
• Lesson 43: do not label metrics with user_id. Label fan-out lag by cohort (normal versus celebrity).
• Lesson 50 and lesson 56, so you do not confuse a feed, a chat thread, and a notification campaign.

Detailed Core Concepts

The product the feed is

TechLearningPro Community has a home feed of accounts you follow, an author profile of one person's posts, and a post page. This lesson designs the home feed and the author profile that the home feed merges. It does not design ads, stories that vanish, or a global "for you" graph over strangers. A ranked "for you" mode may reorder people you already follow. Recommending strangers is a different candidate generator and a different safety review.

A post has an author, a creation time, a visibility (public, followers, author_only), a text body, and optional media object ids. Edits create a new version row; the feed stores the post id and hydrates the current version so an edit does not require a second fan-out of the body. Deletes set removed_at. Removed posts are filtered at hydrate time, which is how a takedown reaches celebrity merges without a million deletes.

Functional requirements

Non-functional requirements

These are drafts for Non-Functional Requirements and SLO Design. A product manager who wants a 50 ms p99 including a remote ranker model has to move the ranker off the critical path or drop it from the SLO.

Fan-out on write, fan-out on read, hybrid

Fan-out on write. On PostCreated, look up followers and append post_id to each follower's timeline structure. The read is a single key lookup. The write cost is proportional to the follower count. Unfollow must delete or tombstone that author's posts in one viewer's timeline, which is bounded. A celebrity unfollow is not a special case if you never wrote those ids out.

Fan-out on read. The home feed queries recent posts for each followee and merges by time. The write is O(1). The read cost is proportional to how many accounts you follow and how you index them. At 80 followees this can be one SQL query (author_id = ANY (?) and created_at < cursor) if the set is small. At 5,000 followees it is a large scatter. It also does not get cheaper as the product adds idle followers who never open the app: you pay on read, and only for people who read. That is why read-fan-out is attractive for celebrities (many followers, the post is read by a fraction) and painful as the only design for everyone.

Hybrid. Maintain follower_count on the author, updated when follows commit, and a boolean celebrity when the count is at least 10,000. The threshold is a configuration value, chosen from the cost of a synchronous fan-out, not a law. Ten thousand Redis writes, pipelined, are a short background task. Five million are a denial of service against your own cluster.

• Under the threshold: publish PostCreated, consumers append the id to each follower's timeline, capped at the most recent 800 ids.
• At or above the threshold: append the id to that author's celebrity list only (the latest 200 ids). Do not walk the follower table.
• On read: load the viewer's timeline ids, load the celebrity lists for celebrities they follow (the set is usually small; cap the merge at 200 celebrity followees and rank which celebrities are in the cap by affinity if a viewer follows more), merge by (created_at, post_id), dedupe, drop tombstones, hydrate bodies, then optionally rerank the first page.

When an account crosses 10,000, new posts stop fanning out. Old ids already in follower timelines age out of the 800 cap. Until they age out, dedupe keeps the read merge from doubling them. A background trim can delete that author's ids from timelines; it is an optimization, not a correctness requirement, because hydrate-time checks still apply.

Backfill on follow of a normal account: a job inserts that author's recent post ids, up to the cap, into the follower's timeline. Backfill on follow of a celebrity: no timeline writes. The next read merges them. This asymmetry is the point of the hybrid. Getting it backwards — backfilling a celebrity into millions of caches when someone famous follows you — is the bug. Backfill walks the author's recent posts into the new follower's one timeline, never the other direction, except for the ordinary fan-out when the new follower later posts.

Ranking

Candidate generation and ranking are separate. Candidates are the merged chronological ids, more than one page worth (for example 200) so the ranker has room. Features that are legitimate here: recency decay, whether the viewer has recently liked or replied to that author, and post type (instructor announcement versus peer note). Features that do not belong in the first version: anything that requires a second network call you cannot time-bound.

The ranker runs with a 50 ms budget. On timeout or error the API returns the chronological merge it already has. Ranked order uses a cursor that includes the sort key and post_id. If the sort key is a score that changes every minute, page two drifts. Freeze the score for a session cache of a few minutes keyed by viewer and mode, and paginate inside that frozen list. When the cache expires, the next "top of feed" is a new ranking. Infinite scroll within a session stays stable. This is a better product than a live score that repeats posts as they bubble up.

Chronological mode skips the ranker entirely. Ship it first. It is the fallback and the debug mode when a learner says "I am not seeing the announcement."

Visibility, block, and mute

Authorization is not a property of the cache. The shared body cache may hold a followers-only post because a follower fetched it. The home-feed API still checks, for each hydrated post: not removed; visibility allows this viewer (public, or followers and a follow edge exists, or the viewer is the author); no block in either direction; no mute from the viewer. Checks are batched: one query for the page's author ids against the viewer's follow, block, and mute sets, which are themselves cached.

Fan-out also applies blocks so we do not fill a timeline with ids we always drop. A block created after fan-out is enforced at read, otherwise a block would require a scan of historical timelines. Read-time enforcement is the correctness net. Fan-out-time enforcement is the cost optimization.

Mutes are read-time only. The muted author is not told. Do not write a mute by deleting ids if the mute is supposed to be reversible without a rebuild; filter instead.

Architecture and Design Diagrams

Posts and follows are the source of truth. Timelines and celebrity lists are Redis structures rebuilt from that truth. The ranker is optional and off to the side.

mermaidCopy
flowchart LR
  subgraph clients [Clients]
    App[Learner app]
  end

  subgraph edge [Edge]
    LB[L7 load balancer]
  end

  subgraph feed [Feed system]
    PostAPI[Post API]
    FeedAPI[Feed API]
    Fan[Fan-out workers]
    DB[(PostgreSQL posts and follows)]
    Redis[(Redis timelines and author lists)]
    Body[Post body cache]
    Rank[Ranker]
  end

  App --> LB
  LB --> PostAPI
  LB --> FeedAPI
  PostAPI --> DB
  PostAPI --> Fan
  Fan --> DB
  Fan --> Redis
  FeedAPI --> Redis
  FeedAPI --> Body
  FeedAPI --> DB
  FeedAPI --> Rank

mermaidCopy
sequenceDiagram
  participant V as Viewer
  participant F as Feed API
  participant R as Redis
  participant DB as PostgreSQL
  participant Rank as Ranker

  V->>F: GET /v1/feed?mode=ranked&cursor=
  F->>R: Timeline ids for viewer
  alt Timeline miss
    F->>DB: Recent posts by non-celebrity followees
    F->>R: Repopulate capped timeline
  end
  F->>R: Celebrity author lists the viewer follows
  F->>F: Merge, dedupe, drop muted and blocked
  F->>DB: Hydrate bodies and visibility for the page
  alt Ranker healthy and mode=ranked
    F->>Rank: Score candidates with 50 ms budget
    Rank-->>F: Order or timeout
  end
  F-->>V: Page and next cursor

The author profile list exists for every author, celebrity or not. The profile page reads it directly and does not consult followers. That path is also the rebuild input.

Technical Explanation

Capacity

Assumptions you can revise in a design review:

• 50 million registered accounts, 8 million daily active viewers.
• Each daily active viewer opens the home feed 15 times: 8e6 * 15 = 120e6 reads per day.
• Peak factor 8 in the evening window: 120e6 / 86400 * 8 ≈ 11,100 reads per second. Design point 20,000 home-feed reads per second.
• 2 million posts per day. Follower distribution: median author 40 followers, mean among non-celebrities 80, and 500 celebrities with between 10,000 and 4 million followers. Most posts are not from celebrities. Assume 2% of posts are celebrity posts: 40,000 celebrity posts per day and 1.96 million ordinary posts.
• Ordinary fan-out: 1.96e6 * 80 ≈ 157 million timeline appends per day, about 1,800 appends per second average. A celebrity post that we mistakenly fanned out to 4 million followers is a single-second demand of millions of writes. That spike, not the 1,800 average, is the outage.
• Timeline entry is a post id and a timestamp, on the order of 32 bytes in a compact list. Cap 800 per viewer: 8e6 daily active * 800 * 32 bytes ≈ 200 GB if every daily active user had a full cap hot at once. Many timelines are shorter. Budget 250 GB of Redis for timelines and author lists, replicated, and treat that as a cost line not a rounding error. Viewers who are inactive are not kept hot; their keys expire after 7 days and rebuild on the next session from PostgreSQL.
• Post bodies: 2 million per day, a few kilobytes each, kept in PostgreSQL and cached when hot. A year of posts is modest compared with timeline amplification. This is why ids are fanned out and bodies are not.
• Follow-edge storage: 50 million users with an average of 80 follows is 4 billion edges if everyone followed that many. Our active community is smaller. Plan 1 billion follow edges, (follower_id, followee_id) primary key plus an index on followee_id for fan-out lookups. PostgreSQL can hold this with partitioning by followee_id hash when the fan-out query is "who follows this author." If the working set outgrows one primary, the follow graph is the first shard candidate, not the post table.

The tier that fails first if you ignore celebrities is Redis and fan-out worker CPU on a single viral post. The tier that fails first on the honest hybrid is home-feed read concurrency on the Feed API and the body hydrate, which scales horizontally because it is stateless aside from Redis. PostgreSQL sees follow-edge reads and post hydrates for cache misses, not 20,000 full merges from disk, as long as celebrity lists and warm timelines hit Redis.

APIs we own

POST /v1/posts with Idempotency-Key, body { "text", "media_ids", "visibility" }. The server assigns post_id and created_at. A replay returns the same post. Media ids must already belong to the author in the file service; the feed does not accept arbitrary URLs.

DELETE /v1/posts/{id} by the author or a moderator role. Sets removed_at and publishes PostRemoved.

POST /v1/follows and DELETE /v1/follows/{followee_id}. Follow is idempotent. A follow of an account that blocked the viewer returns 403 and writes nothing.

GET /v1/feed?mode=chrono|ranked&cursor= returns { "items": [ { "post_id", "author_id", "created_at", "text", "media" } ], "next_cursor" }. The cursor is an opaque encoding of (created_at, post_id) for chronological mode and (session_id, offset_in_frozen_list) for ranked mode. Clients treat it as opaque. GET /v1/authors/{id}/posts?cursor= reads the author list and does not depend on the viewer's timeline.

Rate limits: creating posts, per author, from lesson 35 (for example 10 per minute and 100 per day, with a higher documented limit for verified instructors). Follow creation is limited to slow abuse. Feed reads are limited per viewer high enough that a real client fits and a scraper does not pull the entire graph.

Schema

sqlCopy
CREATE TABLE posts (
  post_id      uuid PRIMARY KEY,
  author_id    uuid NOT NULL,
  visibility   text NOT NULL,
  body         text NOT NULL,
  media_ids    uuid[] NOT NULL DEFAULT '{}',
  created_at   timestamptz NOT NULL,
  removed_at   timestamptz,
  version      int NOT NULL DEFAULT 1
);

CREATE INDEX posts_author_time ON posts (author_id, created_at DESC)
  WHERE removed_at IS NULL;

CREATE TABLE follows (
  follower_id  uuid NOT NULL,
  followee_id  uuid NOT NULL,
  created_at   timestamptz NOT NULL,
  PRIMARY KEY (follower_id, followee_id)
);

CREATE INDEX follows_by_followee ON follows (followee_id, follower_id);

CREATE TABLE blocks (
  blocker_id   uuid NOT NULL,
  blocked_id   uuid NOT NULL,
  created_at   timestamptz NOT NULL,
  PRIMARY KEY (blocker_id, blocked_id)
);

CREATE TABLE mutes (
  muter_id     uuid NOT NULL,
  muted_id     uuid NOT NULL,
  created_at   timestamptz NOT NULL,
  PRIMARY KEY (muter_id, muted_id)
);

CREATE TABLE author_stats (
  author_id       uuid PRIMARY KEY,
  follower_count  bigint NOT NULL,
  celebrity       boolean NOT NULL
);

author_stats.follower_count is updated in the follow transaction. A drift job recounts from follows nightly; the boolean celebrity used for routing can lag by one post without correctness loss because read-path dedupe and visibility checks still hold. Do not fan out based on a count the client sends.

Redis keys, not a second source of truth:

• tl:{viewer_id} a capped list of {post_id, author_id, created_at} for non-celebrity fan-out.
• au:{author_id} a capped list of that author's recent post ids, all authors.
• body:{post_id} cached public body. Followers-only bodies are cached only with a short TTL and still re-checked.

Timeline append uses a Redis pipeline: LPUSH, LTRIM to 800, and an expiry refresh. The operation is idempotent if the worker includes the post id and the reader dedupes; a duplicate LPUSH of the same id is possible with at-least-once delivery and is removed at merge. A Redis set per timeline would make insertion idempotent and complicate "keep the latest 800." Dedupe-at-read is the simpler correctness fix. The trim keeps memory bounded even if duplicates sneak in for a while.

Scale, cache, and load balancing

Feed API instances are stateless behind an L7 load balancer, least-requests, no sticky sessions. Post API is a separate deployment so a feed-read spike does not exhaust author-write connections. Fan-out workers consume PostCreated partitioned by author_id so one author's posts stay ordered. For a non-celebrity the consumer loads follower ids in pages and pipelines Redis writes. Page size a few thousand. A crash mid-fan-out reprocesses the event; duplicate ids are tolerated.

Cache misses on tl:{viewer} rebuild from PostgreSQL: non-celebrity followees' recent posts via posts_author_time, limited per author, merged, trimmed to 800, written back. A cold Redis after a flush is a thundering herd of rebuilds at 20,000 reads per second. Protect the database with a single-flight lock per viewer (SET rebuild:{viewer} NX with a short TTL) so one request rebuilds and others wait or receive a slightly stale chronological merge of celebrity lists only. Also rebuild asynchronously for active users after a failover, using the last day's active set, before you declare the cache warm. Disaster Recovery and Business Continuity expects this warmup, not a hope that Redis replicas never fail.

Do not put the home feed behind a CDN. It is per viewer and per authorization. Public author profiles of celebrities can be cached at the edge with a few seconds TTL because they are not private; invalidate on PostRemoved by author id. A followers-only post on a celebrity is not a CDN object.

Read replicas serve rebuild queries and follower-page scans. The follow transaction and the new post insert use the primary. The viewer's own profile read after a write uses the primary so read-your-writes holds. Home feed of other people's posts may use a replica for rebuild; the Redis timeline, once written by the fan-out worker from the primary's event, is the fresh path.

Security

The viewer is authenticated. Anonymous requests may read public posts on an author profile and receive nothing followers-only. The feed API never returns a body it has not checked against visibility and blocks on this request. Caching the rendered page in a shared key would leak viewer A's feed to viewer B; the only shared objects are public bodies and celebrity public id lists.

Post text is stored as data and rendered as text in clients. If a web client inserts HTML, it encodes the body. Links in posts are not fetched by the feed service (no SSRF from a preview bot in this version). Media ids are authorized by the file service; the feed does not proxy arbitrary URLs.

Blocks are symmetric in effect and asymmetric in storage: only the blocker has the row, and both directions are queried (blocker = me AND blocked = author OR blocker = author AND blocked = me). Follower counts used for celebrity routing are not a secret, but the follower list of a private account is. Fan-out workers are a privileged service. They are not exposed to the client, and their logs contain counts and post ids, not the full follower list.

A removed post stays in PostgreSQL for a moderation retention window with removed_at set, body still present for the trust-and-safety role, and absent from hydrate for everyone else. That is a different visibility rule, implemented as a role check, not as a second cache.

Failure modes

Trade-offs

Threshold value. Lower thresholds fan out less and merge more at read. A viewer who follows 300 "small celebrities" pays 300 list reads, which is still a Redis pipeline, and then a wide merge. Higher thresholds make viral posts expensive. Start at 10,000, measure the p99 fan-out time and the p99 merge fan-in, and move the number. Publish it as configuration.

Redis timelines versus a durable timeline table. A table of (viewer_id, post_id) at 157 million rows per day is a storage product of its own. Redis with a rebuild from posts is the right first system because the feed only shows recent items. Choose a durable timeline store when product requires an infinite scroll through a year of home feed without rebuilding from followees, or when Redis warmup after failover cannot meet the availability SLO. Say the cost before you choose it.

Push bodies versus push ids. Bodies make the read a single lookup and make edits and deletions a second fan-out of the same width as the original post. Ids make hydrate a batch get, which is cheap for a page of twenty, and make edits free. Push ids.

Ranking as the source of truth. Storing only the ranked order means a model rollback changes history and a bug hides the exam post. Store candidates chronologically. Rank in a session-scoped frozen list.

Sync fan-out in the post request. The author would wait on their follower count. Even 80 synchronous Redis writes inside the HTTP call couple post latency to cache health. The post transaction commits the row and the outbox. Workers fan out. The author's profile list can be updated in the post transaction because it is one key and gives read-your-writes on their own profile.

Practical Implementation or Design Walkthrough

A normal post. A learner with 80 followers submits a question. The Post API inserts posts, confirms celebrity = false, and writes an outbox event in that transaction. The worker reads follower ids, pipelines LPUSH and LTRIM on eighty keys, and appends the author list. A follower's next GET /v1/feed hits tl:{viewer}, merges any celebrities they follow, hydrates twenty bodies, and returns a cursor of the last (created_at, post_id).

A celebrity post. An instructor with 4 million followers submits the exam change. The API sees celebrity = true, appends au:{author}, and does not enqueue a per-follower walk. Followers who open the feed pull that author list because the instructor is in their celebrity follow set. Freshness is one Redis write plus replication lag. The notification platform, if product wants a push, sends to people who opted into "instructor announcements," which is a tiny subset and is lesson 56's campaign, not this fan-out.

Unfollow. Delete the follow edge on the primary. If the followee was non-celebrity, a job LREMs that author's ids from the viewer's timeline or, simpler and good enough, sets a tombstone set gone:{viewer} of author ids checked at merge until the timeline is rebuilt. Celebrity unfollow only removes the id from the viewer's celebrity set. No million-key update exists in that path, which is the operational win.

Follow a normal account. Insert the edge, increment follower_count, and enqueue a backfill of the author's recent ids into the one new timeline, capped, skipping removed and invisible posts.

Ranked session. The Feed API builds 200 chronological candidates, calls the ranker with a deadline, stores the frozen id list under rank:{viewer}:{session} for five minutes, and returns the first page. The next cursor carries the session id and the index. If the ranker misses the deadline, the response is chronological and the session cache stores that order so page two does not switch algorithms mid-scroll.

Validation: a test with one celebrity and one normal author asserts the celebrity post caused zero timeline writes and one author-list write; a block created after fan-out disappears on the next read; a Redis flush of one viewer's key rebuilds from SQL; two deliveries of PostCreated produce one visible id; a followers-only post fetched by a stranger returns 404.

Enterprise Real-World Scenarios

Scenario 1: An instructor crosses a million followers during a live course

A learning company runs a public feed beside its courses. One instructor's follower count grows from a few thousand to over a million in a month. Fan-out was implemented as "always walk follows_by_followee." The first post after the million-follower week saturates Redis, fan-out lag for every author climbs because the consumer group is shared, and ordinary study-group posts are minutes late. The home feed looks "down" to small communities that were never the problem.

The fix is the hybrid in this lesson, applied as a migration rather than a rewrite. Authors above the threshold flip celebrity. A deploy stops new fan-out for them. Existing timeline ids age out by trim. Dedupe is shipped before the flag flip so the overlap week is safe. Fan-out lag alerts split by cohort so a celebrity incident cannot hide inside the average. The company also separates the consumer for celebrity author-list updates, which are O(1), from the consumer that walks followers, so a bug that mislabels a celebrity still has a concurrency cap: the walker refuses jobs whose follower page budget exceeds the threshold and sends them to a quarantine topic.

Product learns to ask for freshness on the instructor post, which the author list satisfies, and stops asking for a push to all four million followers. The subset that wants a push is a notification campaign with consent, rated and isolated as in lesson 56.

Scenario 2: A private cohort inside a public community

A university tenant uses the same feed for a followers-only cohort. Students assume posts are invisible to the public MOOC. The implementation caches rendered feed pages in a CDN keyed only by a course id to save origin traffic. A student copies a URL, the CDN serves the page, and the post is public for the TTL.

The review finding is that per-viewer authorization and shared caches do not mix. The repair removes the CDN from the home feed, keeps public author profiles cacheable only when visibility = public, and adds a test that a followers-only hydrate with no follow edge returns 404 even when the body is in Redis. The university's trust document can then say something true: visibility is enforced on every read, and the cache is not the authority. Architecture Security and Threat Modeling uses this as the worked example of a cache that became an access-control bypass. The feed team also partitions metrics and moderator tools by tenant so a takedown in the cohort does not depend on a MOOC moderator's queue.

Best Practices

• Treat posts and follows as the source of truth. Treat timelines as a capped, rebuildable cache.
• Compute amplification at a high percentile of follower count, not at the mean.
• Fan out ids, not bodies. Hydrate a page, not a history.
• Keep an author list for every account. It serves the profile and the celebrity merge.
• Dedupe by post id on every merge. At-least-once fan-out and threshold crossings depend on it.
• Enforce visibility, blocks, and mutes at read. Use fan-out-time filters only as an optimization.
• Paginate with opaque cursors. Freeze ranked sessions.
• Time-bound the ranker and ship chronological mode as the fallback you have already tested.
• Single-flight timeline rebuilds. Warm active keys after a cache failover.
• Do not CDN the home feed. You may cache public celebrity profiles with a short TTL and an explicit invalidation on removal.
• Update celebrity flags from a server-side count inside the follow transaction, then reconcile nightly.
• Rate limit creates. A bot with a normal follower count is still an abuse problem; a bot that is a celebrity is an incident.
• Alert on fan-out lag for non-celebrities and on merge latency separately.
• Keep notification fan-out in the notification platform. The feed shows the post when the app is open.

Common Mistakes and Anti-Patterns

Designing for average followers. The capacity spreadsheet says 1,800 writes per second and the cluster is sized for 5,000. One instructor post asks for millions. The average was never the workload. Put the maximum fan-out you will actually perform in the spreadsheet, and refuse the rest with the celebrity path.

Storing full bodies in every timeline. Edits and takedowns become as expensive as the original post, and Redis holds thousands of copies of the same announcement. Store ids.

Offset pagination. LIMIT 20 OFFSET 10000 on a feed that just received a new top post shifts the page. Users see duplicates and gaps. Cursors of (created_at, post_id) or a frozen session list do not.

Ranker in the only success path. A model deploy that returns errors takes down the feed. The chronological merge must already be in memory and returned on timeout.

Shared rendered-page cache. It is the fastest way to leak a private cohort. Cache ids and public bodies. Authorize on the way out.

Synchronous fan-out in the POST. Author latency tracks follower count and Redis health. Commit the post, fan out asynchronously, and make the author's own profile read the author list updated in the write transaction.

Deleting posts by scanning every timeline. A takedown of a celebrity would be a map-reduce job. Tombstone at the post row and filter at hydrate so the worst case is one write.

Backfill in the wrong direction. "User A gained a follower, copy A's posts to everyone" is how a celebrity follow explodes. Backfill copies the followee's recent ids into the new follower's one timeline, and only when the followee is not a celebrity.

One consumer group for all fan-out. A single hot author blocks the partition. Partition by author, cap the work, and quarantine anything above the threshold.

Troubleshooting and Design Review

Review checklist for Enterprise Architecture Review Checklist:

• Amplification computed for a celebrity, with the threshold written down.
• Rebuild path named, including herd protection.
• Authorization on every hydrate, independent of cache hits.
• Cursor strategy for both modes.
• Ranker timeout behavior demonstrated in a test.
• Deletes are tombstones, not cluster-wide scans.
• Notification fan-out is not smuggled into the timeline design.

Interview Questions and Answers

Q1. Walk through fan-out on write versus fan-out on read. Which do you pick?

Fan-out on write does the follower walk when the post is created and makes the home-feed read a lookup of one key. It costs writes proportional to followers and makes a four-million-follower post a self-inflicted overload. Fan-out on read stores the post once and merges followees at read time. Writes stay cheap and reads grow with how many accounts the viewer follows, paid only when someone actually opens the app. For TechLearningPro's skewed graph I use both. Under 10,000 followers, fan out ids into a capped per-viewer timeline. At or above that, append the author's own recent-id list and merge those lists at read for the celebrities this viewer follows. I dedupe by post id. I do not claim one strategy is universally faster; I show the write spike the pure write strategy cannot absorb and the read fan-in the pure read strategy imposes on a viewer who follows thousands of ordinary accounts.

Q2. How do you stop a celebrity post from taking down ordinary feeds?

The post path reads author_stats.celebrity, which flips from the follower count maintained in the follow transaction. Celebrity posts do not enqueue a follower walk. They update one author key. Fan-out workers also refuse a job whose follower count exceeds the threshold and quarantine it, so a stale flag cannot schedule millions of writes. Ordinary authors use a different lag alert. Sharing one unbounded worker pool means the celebrity's queue time is everyone else's freshness SLO. The read path still shows the post because every home feed merge includes the viewer's celebrity set.

Q3. What do you store in the timeline, and how is a delete handled?

Post id, author id, and created_at. Not the body. A delete sets removed_at on the post row and publishes an event that can LREM the id from the author list. Follower timelines may still contain the id until trim or rebuild. Hydrate drops removed ids, so the takedown is one row plus a filter, not a scan of every follower. Edits bump version on the same row and the body cache key includes the version or is deleted on write. Old timeline ids automatically show the new text. That is the payoff for storing ids.

Q4. How do you paginate a feed that is receiving new posts?

Chronological pages use an opaque cursor of the last item's created_at and post_id, and the query or the list walk asks for items strictly older than that tuple. A new post at the top does not shift page two. Large SQL offsets do, and they get more expensive as the offset grows. Ranked pages freeze about 200 ids for a few minutes under a session key and paginate by index inside that list, because a live score reshuffles items under the user. When the session expires, the next refresh is a new ranking, which is what "pull to refresh" means.

Q5. Where does ranking live, and what happens when it fails?

Ranking consumes a candidate list the chronological merge already produced. It does not generate candidates in the first design, so it cannot invent posts the viewer does not follow. It runs under a 50 millisecond deadline. On timeout the API returns the chronological order and still caches that order for the session so the algorithm does not change between page one and page two. I would rather ship a correct chronological feed than a ranked feed whose availability equals the model's availability. Features stay cheap: recency, affinity to that author, post type. If a feature needs a second fan-out to a feature store, it is not on the critical path until its own latency budget is proven.

Q6. The timeline cache is empty after a failover. What happens to posts?

Nothing is lost. Posts and follows are in PostgreSQL. The next read for a viewer single-flights a rebuild: recent posts from non-celebrity followees, trimmed to the cap, written back to Redis. Other requests for that viewer wait on the lock instead of stampeding the database. Celebrity lists are separate keys and are cheap to rebuild from each author's posts. A warmup job rebuilds timelines for yesterday's active viewers before traffic returns to full weight. I size the database for rebuild rate, not only for steady-state hydrate misses. If product cannot accept a cold-cache latency hit, the timeline has to be durable, and that is a different cost I would estimate before promising it.

Q7. How do blocks and private posts stay correct if ids are already cached?

Every hydrate batch-checks visibility, follow edges for followers-only posts, blocks in both directions, and mutes. The timeline is a hint about what might be relevant, not a grant of access. A block after fan-out relies on that check immediately; a background purge can clean the ids later. Followers-only bodies are not placed on a CDN. A leaked post id returns 404 to a viewer with no edge. Tests cover the stranger, the follower, the blocked viewer, and the author. The author can see author_only drafts; nobody else's cache entry is sufficient.

Q8. How do you estimate Redis memory and write rate?

Pick daily active viewers, cap per timeline, and bytes per id. Eight million viewers times 800 ids times about 32 bytes is on the order of 200 GB plus overhead, and inactive keys expire so the hot set tracks recent viewers rather than all 50 million accounts. Write rate is ordinary posts times their follower counts, not all posts times the largest follower count, once celebrities are excluded. I state both numbers. I also state the spike I refused: a four-million-follower fan-out that the design does not perform. Memory is why the cap exists. Without LTRIM, a year of appends is the real bill and the list is no longer a cache.

Q9. What is the freshness SLO for a classmate's post versus an instructor's post?

Classmate, non-celebrity: the post commits, the outbox publishes, the worker pipelines Redis appends. The SLO is about two seconds p99 from commit to visibility on a follower's next fetch, dominated by queue lag. I measure it with a synthetic follower. Instructor, celebrity: one author-list append, visibility on the next merge, about one second p99, dominated by Redis replication and client poll interval. I do not promise push delivery to all followers; that is a notification campaign with consent and its own SLO. Profile reads of the author's own post are read-your-writes via the author list updated on the write path.

Q10. A viewer follows 150 celebrities and 80 classmates. What does the read do?

Load one classmate timeline key. Pipeline-get the celebrity author lists. If 150 is under the merge cap of 200, merge all of them by time, dedupe against the timeline, filter mute and block, take the candidate window, hydrate the page. If a viewer follows more celebrities than the cap, keep the cap and choose which lists to merge by a stored affinity so the read cost stays flat. The classmate timeline is O(1) regardless. This is why the hybrid still needs a cap on the read side: a user who follows every instructor in the catalog must not turn every home-feed request into thousands of key lookups. The product can also offer per-instructor views, which are a single author list and do not belong in the home merge.

Q11. How do you make fan-out idempotent?

PostCreated is at least once. The consumer may run twice. Timeline structures can contain a duplicate id; the merge step drops duplicate ids before hydrate, so the client does not see two copies. The author-list append has the same rule. Follow backfill uses the same append path. I do not depend on exactly-once queue processing. If I need stronger write idempotency for cost, a set of recent ids per viewer with a short TTL can reject duplicates before LPUSH, at the price of a second structure. Correctness starts at the merge, not at the broker.

Q12. What would you change for a hundred times the read rate, and what would you not change?

I would add Feed API replicas and Redis read replicas, shard Redis by viewer id, and shard the follow graph by followee id so fan-out page scans stay local. I would keep the hybrid threshold, the id-not-body rule, cursors, read-time authorization, and the ranker timeout. I would not "solve" scale by fanning out celebrity bodies into a bigger cluster. A hundred times the followers on one instructor is the same celebrity problem with more zeros, and the author-list design already isolates it. I would revisit a durable timeline only if the warmup story cannot meet the availability SLO at that read rate. Architecture Performance and Capacity Review is the place to show the new shard map. The capstone can add a stranger-recommendation candidate source; it should not replace the follow graph as the source of truth.

Hands-On Exercises

Exercise 1: Size the hybrid

Problem. 8 million daily active viewers, 15 feed opens each, peak factor 8. 2 million posts a day. One instructor has 4 million followers. Ordinary authors average 80 followers. Threshold 10,000. Timeline cap 800 ids at 32 bytes.

Tasks.

1. Compute steady-state home-feed reads per second and the design point you would actually provision.
2. Compute ordinary fan-out writes per second, and the spike if the instructor were fanned out on write.
3. Estimate hot timeline memory.
4. Specify Redis keys and the read merge, including dedupe and a cap on celebrity lists per viewer.
5. Specify cursor behavior for chronological mode.
6. Specify rebuild single-flight after a flush.
7. Specify the block check that still runs when the id is cached.
8. State the ranker timeout and the response when it fires.

Deliverables. The arithmetic, a sequence diagram, and a short failure note for the instructor post.

Exercise 2: Review a pure write-fan-out design

The proposal stores the full JSON post on every follower's timeline, paginates with OFFSET, runs the ranker as a required dependency, and serves the rendered home page from a CDN keyed by user id that a teammate later "simplifies" to a course id. Write the review. Include the celebrity incident, the edit problem, the pagination bug, the ranker outage, and the privacy bug.

Rubric.

Lesson Summary

The home feed is a derived, capped, authorized view over posts and follows. Fan-out on write serves ordinary accounts because their follower counts are a bounded Redis pipeline and the read becomes one key. Fan-out on read serves celebrities because walking millions of followers is an outage. The hybrid uses a server-side threshold, an author list for every account, a merge that dedupes, and read-time checks for visibility, blocks, and mutes. Ranking reorders a frozen candidate list and loses to the chronological merge when it is slow. Cursors replace offsets. Timeline loss is a latency event with a single-flight rebuild, not a data loss, because the source of truth was never Redis.

Notification campaigns and object storage remain separate systems. Module 6's interview, SLO, capacity, security, and disaster-recovery lessons are where you defend the threshold, the warmup plan, and the decision that a shared cache does not grant access.

Further Reading

• Caching Strategies and Cache Invalidation, Database Design and Indexing for Scale, and Message Queues and Event Streaming.
• Design a Notification Platform and Design a Cloud File Storage System for the neighboring fan-out and media paths.
• System Design Interview Framework, Non-Functional Requirements and SLO Design, Architecture Security and Threat Modeling, and Disaster Recovery and Business Continuity.
• Redis list commands LPUSH, LTRIM, and LREM, used as the capped timeline structure: https://redis.io/docs/latest/commands/ltrim/.
• PostgreSQL indexes and partial indexes, for the author timeline query: https://www.postgresql.org/docs/current/indexes.html.
• Redis SET with NX and expiry, the single-flight lock primitive: https://redis.io/docs/latest/commands/set/.