News feed: push, pull or both · Design Patterns in Practice

Fan-out on write, fan-out on read and the hybrid most large feeds end up with, simulated in TypeScript so you can see where each one spends its work.

The Observer chapter ended with a subject notifying its subscribers in memory. A news feed is the same idea at the scale of a company: when someone posts, everyone who follows them should eventually see it. The design question is when the work of delivering a post happens, because that one choice decides how fast writes are, how fast reads are, and what breaks when a celebrity with ten million followers posts a photo.

There are two pure answers and one practical one:

Pull (fan-out on read): do nothing when a post is written; build each feed when it is requested.
Push (fan-out on write): when a post is written, copy a reference to it into every follower's feed right away.
Hybrid: push for normal accounts, pull for the few accounts with huge audiences.

The model

A feed needs three pieces: who follows whom, the posts themselves, and a strategy for turning those into a feed. The strategy is literally the Strategy pattern from chapter 13, which makes it easy to swap and compare.

type Post = { id: number; authorId: string; text: string; createdAt: number };

class SocialGraph {
  private following = new Map<string, Set<string>>();
  private followers = new Map<string, Set<string>>();

  follow(follower: string, followee: string) {
    if (follower === followee) return;
    (this.following.get(follower) ?? this.following.set(follower, new Set()).get(follower)!).add(followee);
    (this.followers.get(followee) ?? this.followers.set(followee, new Set()).get(followee)!).add(follower);
  }
  followeesOf(userId: string) {
    return [...(this.following.get(userId) ?? [])];
  }
  followersOf(userId: string) {
    return [...(this.followers.get(userId) ?? [])];
  }
  followerCount(userId: string) {
    return this.followers.get(userId)?.size ?? 0;
  }
}

class PostStore {
  private byAuthor = new Map<string, Post[]>();
  private nextId = 1;
  private clock = 0;
  scanned = 0; // how many posts feed reads had to look at

  create(authorId: string, text: string): Post {
    const post = { id: this.nextId++, authorId, text, createdAt: ++this.clock };
    (this.byAuthor.get(authorId) ?? this.byAuthor.set(authorId, []).get(authorId)!).push(post);
    return post;
  }
  recentBy(authorId: string, limit: number): Post[] {
    const posts = (this.byAuthor.get(authorId) ?? []).slice(-limit).reverse();
    this.scanned += posts.length;
    return posts;
  }
}

interface FeedStrategy {
  onPost(post: Post): void;
  feedFor(userId: string, limit: number): Post[];
}

const newestFirst = (posts: Post[]) => [...posts].sort((a, b) => b.createdAt - a.createdAt);

In production the graph would be a table or a graph store, the posts a partitioned database, and the per-user feeds a Redis list or sorted set. The shapes are the same.

Pull: build the feed on read

class PullFeed implements FeedStrategy {
  constructor(
    private readonly graph: SocialGraph,
    private readonly posts: PostStore,
  ) {}

  onPost() {} // nothing to do at write time

  feedFor(userId: string, limit: number) {
    const candidates = this.graph.followeesOf(userId).flatMap((id) => this.posts.recentBy(id, limit));
    return newestFirst(candidates).slice(0, limit);
  }
}

Writes are as cheap as they can be: one insert. Reads do all the work: look up every account the user follows, fetch their recent posts, merge and sort. For someone following 2,000 accounts, that is 2,000 lookups on every refresh, and the feed is only as fast as the slowest partition you asked.

Pull has real advantages. Nothing is precomputed, so nothing is wasted on users who never open the app, and the feed always reflects the latest state (deleted posts, blocked accounts, new ranking signals). That is why ranked feeds lean towards assembling candidates at read time: Facebook has described its feed as aggregated per request from servers that hold recent activity, then ranked.

Push: deliver on write

class PushFeed implements FeedStrategy {
  private inboxes = new Map<string, Post[]>();
  writes = 0; // how many feed entries fan-out had to write

  constructor(
    private readonly graph: SocialGraph,
    private readonly maxFeedLength = 500,
  ) {}

  onPost(post: Post) {
    for (const follower of this.graph.followersOf(post.authorId)) {
      const feed = this.inboxes.get(follower) ?? [];
      feed.unshift(post);
      if (feed.length > this.maxFeedLength) feed.pop();
      this.inboxes.set(follower, feed);
      this.writes++;
    }
  }

  feedFor(userId: string, limit: number) {
    return (this.inboxes.get(userId) ?? []).slice(0, limit);
  }
}

Now a read is a single key lookup, which is what you want when reads outnumber writes 25 or 100 to one. The cost moved to the write path: a post by an account with N followers means N writes. In Redis that is LPUSH feed:{followerId} {postId} followed by LTRIM feed:{followerId} 0 499 for every follower, done by workers reading from a queue so the author's request returns immediately.

Two problems show up as you grow:

The celebrity problem. An account with 10 million followers turns one post into 10 million writes. Fan-out falls behind, followers see the post minutes late, and the queue backs up for everyone else.
Wasted work. Most followers of a big account will not open the app today, but you wrote to their feed anyway.

Twitter's well-known timeline design is push-based for exactly the read-latency reason, with a separate path for accounts with very large follower counts.

Hybrid: push for most, pull for the few

class HybridFeed implements FeedStrategy {
  private readonly push: PushFeed;

  constructor(
    private readonly graph: SocialGraph,
    private readonly posts: PostStore,
    private readonly celebrityThreshold = 10_000,
  ) {
    this.push = new PushFeed(graph);
  }

  get writes() {
    return this.push.writes;
  }

  private isCelebrity(userId: string) {
    return this.graph.followerCount(userId) >= this.celebrityThreshold;
  }

  onPost(post: Post) {
    if (!this.isCelebrity(post.authorId)) this.push.onPost(post);
  }

  feedFor(userId: string, limit: number) {
    const pushed = this.push.feedFor(userId, limit);
    const pulled = this.graph
      .followeesOf(userId)
      .filter((id) => this.isCelebrity(id))
      .flatMap((id) => this.posts.recentBy(id, limit));
    return newestFirst([...pushed, ...pulled]).slice(0, limit);
  }
}

Normal accounts fan out on write, so most of a feed is precomputed. Celebrity posts are not copied anywhere; each reader merges in the recent posts of the handful of celebrities they follow at read time. A user typically follows a few big accounts, not thousands, so the read stays cheap.

Measuring the difference

Here is a small world: 50,000 users who each follow 20 others, all of whom also follow one big creator. The creator posts once, 1,000 random users post once each, then 1,000 random users load their feed.

function simulate(name: string, make: (graph: SocialGraph, posts: PostStore) => FeedStrategy & { writes?: number }) {
  const graph = new SocialGraph();
  const posts = new PostStore();
  const users = 50_000;
  for (let i = 0; i < users; i++) {
    graph.follow(`user-${i}`, "big-creator");
    for (let f = 1; f <= 20; f++) graph.follow(`user-${i}`, `user-${(i * 7 + f * 13) % users}`);
  }

  const feed = make(graph, posts);
  let seed = 42;
  const random = () => (seed = (seed * 16807) % 2147483647) % users;

  feed.onPost(posts.create("big-creator", "new single out now"));
  for (let i = 0; i < 1_000; i++) feed.onPost(posts.create(`user-${random()}`, "hello"));

  let sawCreator = 0;
  for (let i = 0; i < 1_000; i++) {
    const items = feed.feedFor(`user-${random()}`, 20);
    if (items.some((p) => p.authorId === "big-creator")) sawCreator++;
  }

  console.log(name.padEnd(6), { fanOutWrites: feed.writes ?? 0, postsScannedOnRead: posts.scanned, sawCreator });
}

simulate("pull", (graph, posts) => new PullFeed(graph, posts));
simulate("push", (graph) => new PushFeed(graph));
simulate("hybrid", (graph, posts) => new HybridFeed(graph, posts));

The output (the random generator is seeded, so it is the same on every run):

pull   { fanOutWrites: 0, postsScannedOnRead: 1426, sawCreator: 1000 }
push   { fanOutWrites: 69999, postsScannedOnRead: 0, sawCreator: 1000 }
hybrid { fanOutWrites: 19999, postsScannedOnRead: 1000, sawCreator: 1000 }

All three strategies return exactly the same feeds; I checked by hashing every feed they produced. What changes is where the work lands:

Pull writes nothing, but each read looks up all 21 accounts the reader follows and scans their recent posts. With real follow counts in the hundreds, that number explodes, and each account looked up is a network call.
Push makes reads free, but the single creator post alone cost 50,000 writes, two and a half times the roughly 20,000 writes of the thousand normal posts combined.
Hybrid keeps the cheap fan-out for normal posts, skips the 50,000-write burst entirely, and pays one extra lookup per read for the one celebrity each user follows.

Change celebrityThreshold, the number of follows or the read/write mix and run it again. Picking the threshold is a real tuning decision: it is the point where one post's fan-out costs more than all the extra read-time merges it saves.

Details that matter in production

Store IDs, not posts. Feeds hold post IDs (and maybe a score); the post body is fetched and cached separately, so an edit or delete does not have to touch millions of feeds.
Fan out asynchronously. The write request publishes a PostCreated event; workers do the fan-out from a queue, which gives you retries, backpressure and a way to see lag. This is Observer across services.
Skip inactive users. Do not fan out to accounts that have not been active for weeks. When they come back, build their feed with a pull once and resume pushing.
Cap feed length. Nobody scrolls past a few hundred items in a precomputed feed; older content can come from a pull path.
Ranking changes the shape. A chronological feed fits push well. A ranked feed needs many candidates scored with fresh signals at read time, which pulls the design back towards read-time aggregation, often with a push-based candidate list as one of the inputs.
Deletes and blocks. Filter at read time even with push, because a precomputed feed can contain posts that became invisible after they were delivered.

Exercises

In simulate, make 1% of users celebrities with 5,000 to 50,000 followers each. Find the celebrityThreshold that minimises fanOutWrites + postsScannedOnRead.
Add deletePost(postId) and make sure deleted posts never appear in any strategy's feed, without scanning every inbox.
Replace PushFeed's in-memory map with Redis (LPUSH, LTRIM, LRANGE) and move onPost into a worker that consumes a queue.
Add an activeSince timestamp per user and skip fan-out for users inactive for 30 days. What has to happen when they log in again?