Design a News Feed

Problem statement

The prompt is to design a news feed for a social service. A news feed is a list of recent posts from the accounts a user follows, shown newest first. A post is a short message a user publishes for followers to see, and a follow is a link from one user to an account whose posts that user wants in the feed. The core loop is plain to state and hard to serve: a user follows accounts, those accounts post, and the feed shows their recent posts in time order.

One word names the central mechanism before any strategy is chosen. Fan-out means spreading a new post out towards the followers who should see it, so posting fans a post out to followers' feeds in one form or another. The service is a large consumer product rather than a small site, so the feed has to stay quick while posting and reading both run constantly through the day. Freshness is part of the product itself, because a new post should reach followers' feeds quickly rather than in a later batch.

Three small models pin down the vocabulary before scale enters the picture. The follow graph stores who follows whom, and every later decision reads that graph in one direction or the other. Posts carry the time they were posted, because newest-first order is a sorting by that time and nothing else. Posting, at its smallest, stores one post under its author, and any spreading towards followers is extra work on top of that single store.

follows = {
    "ana": ["cy"],
    "ben": ["ana"],
    "cy": ["ana", "ben"],
}

for user in sorted(follows):
    print(user, "follows", ", ".join(follows[user]))

The graph in the model is deliberately tiny, with each user following a handful of accounts. Even at that size the direction matters, because following is one-way: an account can appear in a reader's feed without that reader appearing in the account's own feed. A reader's feed is gathered from exactly the accounts in that reader's follow list. Nothing outside the list contributes, however recent its posts happen to be.

posts = [
    (5, "ana: morning run"),
    (9, "ana: new photo"),
    (7, "ben: hello all"),
]

for when, text in sorted(posts, reverse=True):
    print(when, text)

Time order is the default order of the feed, called chronological order. Each post carries its posting time, and sorting by that time in reverse puts the newest post first. No judgement about quality enters the ordering at this stage, so a weak new post outranks a strong older one. That plain rule is the baseline against which every ranked alternative is measured.

posts = {"ana": ["ana: morning run"], "ben": [], "cy": []}

posts["ben"].append("ben: hello all")

for author in sorted(posts):
    print(author, "stored posts:", len(posts[author]))

Storing a post once, under its author, is the cheapest possible posting step. The stored counts show one new post changing exactly one author's list and no other structure at all. Every feed strategy starts from that stored post and then differs in what else happens around it. The difference between strategies is therefore not whether a post is stored, but when and how often its presence spreads.

follows = {"ana": ["cy"], "ben": ["ana"], "cy": ["ana", "ben"]}
posts = [
    (5, "ana", "ana: morning run"),
    (9, "ana", "ana: new photo"),
    (7, "ben", "ben: hello all"),
    (6, "cy", "cy: my own post"),
]

feed = [p for p in posts if p[1] in follows["cy"]]
feed.sort(reverse=True)

for when, author, text in feed:
    print(text)
print("cy feed has", len(feed), "posts")

The closing model gathers a feed for one reader from the stored posts and the follow graph together. The reader's own post stays out of the result, because that reader does not follow their own account. The gathered posts are then sorted newest first, matching the definition the feed promises to every reader. Even a gather this small shows the shape of a whole strategy, one where the feed is built at the moment it is read.

In scope

• Posting: a user publishes a short post, stored once under its author and then made visible to followers
• Following and unfollowing: a follow link decides whose posts join a reader's feed, and removing the link stops later posts from joining
• Viewing the home feed: a reader opens the service and receives recent posts from followed accounts, newest first
• Ordering the feed: newest first by default, with a scored order considered as an ordering layer on top

Out of scope

• Search across posts or accounts
• Messaging between users
• Profiles and account pages beyond the follow links the feed needs
• Advertising and promoted posts

Requirements

Requirements turn the bare prompt into a contract the rest of the design can be judged against. Each functional requirement names a behaviour the system must show, together with the consequence that makes it a requirement rather than a wish. Each non-functional requirement names a bar that behaviour must clear under load. The bars that matter most here are cheap reads, freshness, and a posting path that never loses a post it has accepted.

Functional requirements

• Post a post: a user publishes a short post, the post is stored once under its author, and a confirmed post is never lost afterwards
• Follow an account: following adds that account's later posts to the reader's feed, and unfollowing stops later posts from joining, without rewriting posts already seen
• Read the home feed: a reader opens the feed and receives recent posts from followed accounts, newest first, as one ready page rather than a long assembly job
• Include huge accounts: posts from accounts with very large audiences still reach their followers' feeds, even though those posts cannot be spread the ordinary way
• Order the feed deliberately: the feed's order is a stated choice — newest first, or a scored order — and the choice is applied the same way on every read

Non-functional requirements

• Cheap reads at volume: feed reads arrive at 11,574 per second, so a read must stay close to a lookup and must never become the expensive step of the design
• Freshness: a new post reaches followers' feeds quickly, within a short delay rather than a batch cycle, because a feed that lags badly reads as broken
• Posting stays light for the author: publishing costs the author one quick request, however large the audience, and the spreading work happens away from that request
• No wasted work dominates: copies made for followers who never open the feed, and gathering done again on every read, are both costs the design must keep bounded
• Graceful degradation: when part of the system falls behind, feeds grow staler before they fail, and a reader still receives a feed rather than an error

The tension inside the contract is easy to name and impossible to remove. Cheap reads push the work towards post time, cheap posting pushes the work towards read time, and freshness limits how long either side may defer its share. Every strategy considered later is a different answer to that single tension. The design therefore has to be judged as a placement of work, not as a search for a placement with no work at all.

Scale estimates

The estimates start from stated habits, and every rate that follows is derived from those habits in the open. The stated habits are these: 100,000,000 people use the service each day, each user posts 2 times a day, each user checks the feed 10 times a day, and the average account follows 200 accounts. One day holds 86,400 seconds, and dividing a daily count by 86,400 turns it into a per-second rate. Those few inputs produce every number the architecture has to survive.

The scale of the problem

Posting is the first chain of arithmetic, and it is the calmest of the three. The daily post count is 100,000,000 users × 2 posts each = 200,000,000 posts a day. Dividing by the seconds in a day gives 200,000,000 ÷ 86,400 = 2,315 posts per second arriving at the service. That rate is unremarkable on its own, and any design that struggled with posting alone would be struggling with the wrong problem.

daily_users = 100000000
posts_each = 2
posts_day = daily_users * posts_each

print("posts per day:", posts_day)
print("posts per second:", round(posts_day / 86400))

The model computes the same chain in code and prints both the daily count and the per-second rate. Nothing in the model rounds early or hides an input, so the printed rate can be checked against the stated habits by eye. The point of showing the working is that the rate, not the habit, is what sizes machines. A designer who quotes habits in an interview still has to land on the rate before choosing anything.

daily_users = 100000000
reads_each = 10
reads_day = daily_users * reads_each

print("reads per day:", reads_day)
print("reads per second:", round(reads_day / 86400))

Reads form the second chain, and they already run well ahead of posts. The daily read count is 100,000,000 users × 10 checks each = 1,000,000,000 feed reads a day. The same division gives 1,000,000,000 ÷ 86,400 = 11,574 reads per second, each one expecting a ready page of recent posts. A read path that assembles a feed from scratch would repeat that assembly at exactly this rate, all day, which is why reads need a cheap path of their own.

posts_day = 200000000
posts_second = round(posts_day / 86400)
followers = 200

print("feed writes per second:", posts_second * followers)
print("feed entries per day:", posts_day * followers)

Copying is the third chain, and it dwarfs the other two. One post under fan-out on write lands in 200 feeds, one for each account the average reader follows in reverse — that is, one per follower of the author. The write rate becomes 2,315 posts per second × 200 = 463,000 feed writes per second, sustained. Across a day those copies make 200,000,000 × 200 = 40,000,000,000 feed entries, a storage and writing burden created entirely by the copying step.

daily_users = 100000000
posts_day = daily_users * 2
reads_day = daily_users * 10
posts_second = round(posts_day / 86400)
reads_second = round(reads_day / 86400)
writes_second = posts_second * 200

print("posts per day:", posts_day)
print("posts per second:", posts_second)
print("reads per day:", reads_day)
print("reads per second:", reads_second)
print("feed writes per second:", writes_second)

rates = [("posts per second", posts_second), ("reads per second", reads_second), ("feed writes per second", writes_second)]
name, value = max(rates, key=lambda r: r[1])
print("largest rate:", name, value)

The closing model puts all three rates side by side and names the largest. Posts arrive at 2,315 per second, reads arrive at 11,574 per second, and feed writes under full copying reach 463,000 per second. The largest rate is not a read or a post but the internal copying between them, which no user ever requests directly. That inversion is the whole estimation lesson: the decisive load of a news feed is self-inflicted, and it is set by where the design chooses to place the fan-out work.

Two consequences follow before any component is named. First, whichever side carries the fan-out work — post time or read time — that side needs the serious engineering, because the raw post rate of 2,315 per second needs almost none. Second, the average hides a tail, since the average account follows 200 accounts while a celebrity account is followed by 50,000,000 people. A strategy tuned only to the average will meet the tail on the first celebrity post, so the tail gets a treatment of its own later in the design.

API design

A small set of operations carries the whole design, and each operation maps to one functional requirement. Posting stores a post, following maintains the follow graph, and reading returns one page of a reader's feed. The shapes below keep the feed response light: a page carries post references and their text in feed order, never a second copy of the follow graph. Everything slow that a post sets in motion leaves through a queue instead of holding the author's request open.

Core operations

The split between synchronous and queued work follows the reader's and the author's patience. Storing the post, recording a follow and returning a feed page all finish inside the request, because a caller is waiting on each result and the work is one store or one ready list. Copying a post into followers' feeds is queued, because the copying spreads one event into many small writes whose completion no single caller is watching. A post therefore reports success once the post itself is safely stored, and the copies catch up a moment later.

Pagination keeps a feed read bounded however long the underlying list grows. The reader asks for one page with a cursor — an opaque marker naming where the previous page stopped — and the service returns that page plus the next cursor. A page is a fixed slice of the feed order, so a slow reader paging back through older posts never forces the service to rebuild the whole list. The same cursor shape serves the ranked order, because ranking changes the order of the list, not the way a reader walks through it.

Data model

Three entities carry the design, and each one exists because a named flow reads it in a named shape. The post is stored once, the follow graph is read in both directions, and the feed list — where the design keeps one — is a stored list of post references per reader. Keeping the feed entry a reference rather than a copy of the post text matters, because fan-out copies the entry once per follower and every field on it would be copied just as many times. The model below is deliberately small, since the difficulty of a news feed lives in movement, not in variety.

Core entities

The follow graph is the one structure both strategies share, and it is read in opposite directions by the two of them. Fan-out on write reads it backwards at post time, from an author to the author's followers, to know where copies belong. Fan-out on read reads it forwards at read time, from a reader to the accounts that reader follows, to know whose posts to gather. Indexing the follow rows both ways is therefore not an optimisation but a requirement, and the cost of maintaining the second index is paid once per follow rather than once per post or per read.

Denormalisation — storing data in the shape a read needs, instead of only in the shape a write prefers — is the feed list itself. A feed entry duplicates a post reference into a reader-shaped list so that reading stays a lookup, and the price is the write volume the estimates exposed. The post text is never denormalised, because text is the largest part of a post and the reference reaches it with one extra lookup per displayed post. Where a feed list is absent, under fan-out on read, the author's post list is the only stored order and every feed is derived from it on demand.

Partitioning follows the reader and the author, never the post alone. Sharding means splitting a store across many machines by a key, each machine holding the rows for one slice of the key space. Feed lists shard naturally by reader id, because a read touches exactly one reader's list, and author post lists shard by author id, because a pull touches the lists of the accounts one reader follows. A single very large author's list stays on one shard under that scheme, which is acceptable because the list is read as a short newest-first slice rather than in full.

High-level architecture

Requests enter through app servers that hold no durable state of their own, so any app server can take any request and the durable truth lives in the stores below them. The post store holds each post once under its author, and the follow graph sits beside it as the record of who follows whom. The feed store holds the ready-made feed lists built by fan-out, and a cache in front of it serves the hottest lists, because the most active readers ask for their first page far more often than any other slice. A queue with fan-out workers carries the copying work away from the posting request, so a post returns quickly while its copies spread in the background.

+----------+   post / follow   +------------------+
|  Client  | ----------------> |   App servers    |
+----------+                   +------------------+
     ^                          |        |        |
     | feed page                v        v        v
     |                    +----------+ +-------+ +--------+
     +------------------- | Feed     | | Post  | | Queue  |
                          | store    | | store | +--------+
                          +----------+ +-------+     |
                               ^                     v
                               | copies        +----------+
                               +-------------- | Fan-out  |
                                               | workers  |
                                               +----------+

The posting flow

1. The client sends the post text to an app server, which stores the post once under its author in the post store
2. The app server places a fan-out job on the queue and reports success to the author
3. A fan-out worker reads the author's followers from the follow graph
4. For an ordinary author, the worker copies the post reference into the feed list of every follower in the feed store
5. For a huge author, no copies are made, and the post waits on the author's own post list for readers to pull

The reading flow

1. The client asks an app server for one page of the home feed
2. The app server reads the reader's ready-made feed list, cache first and feed store on a miss
3. It pulls the latest posts of any huge accounts the reader follows from their author post lists
4. It merges the pulled posts into the ready list in feed order, applies the chosen ordering, and returns one page
5. Later pages repeat the same steps with the page cursor the previous response returned

Every functional requirement lands on a component in the diagram, and no component is present without a requirement behind it. Posting lands on the app servers and the post store, following lands on the follow graph, and the home feed lands on the feed store, the author post lists and the merge step at read time. Ordering lands at the point a page is assembled, which is why the same ordering choice works over both the ready list and the pulled posts. Freshness lands on the queue: the gap between a post being stored and its copies landing is exactly the queue's delay, so freshness is operated by watching that queue rather than by hoping.

Deep dive: fan-out on write or fan-out on read?

Every news feed design is, at its core, one decision: when is a reader's feed built? The work cannot be avoided, because a feed is a merge of many authors' posts into one reader's order, and that merge happens either before the reader arrives or while the reader waits. The estimates fix the stakes precisely: posts arrive at 2,315 per second and reads arrive at 11,574 per second, so the side that carries the work carries it at very different rates. Two strategies answer the question, they are exact mirrors of each other, and each is genuinely right for some service.

Two ways to build a feed

Fan-out on write: the push model

Fan-out on write means a new post is copied into the feed of every follower at the moment it is posted, a strategy also called the push model. Each user owns a stored feed list that holds the copies made for that user, so the list is built gradually as followed accounts post. Posting cost is set by the audience: one post costs one feed write for every follower of its author. Reading gains the mirror benefit, because opening a feed reads a ready list and a read does almost no work.

followers = {"ana": ["ben", "cy"], "ben": ["cy"], "cy": ["ana"]}
feeds = {"ana": [], "ben": [], "cy": []}

writes = 0
for follower in followers["ana"]:
    feeds[follower].append("ana: sunny day")
    writes += 1

for user in sorted(feeds):
    print(user, "feed:", feeds[user])
print("feed writes:", writes)

The first model pushes a single post and counts what it writes. One post from one author lands in the feed of each of that author's followers, and the author's own feed receives nothing from it. The write count equals the follower count exactly, with no other factor involved. That equality is the push model's whole character: audience size and posting cost are the same number wearing two names.

followers = {"ana": ["ben", "cy"], "ben": ["cy"], "cy": ["ana"]}
feeds = {"ana": [], "ben": [], "cy": []}
posts = [("ana", "ana: sunny day"), ("ben", "ben: game tonight"), ("cy", "cy: new recipe")]

writes = 0
for author, text in posts:
    for follower in followers[author]:
        feeds[follower].append(text)
        writes += 1

for user in sorted(feeds):
    print(user, "feed:", feeds[user])
print("feed writes:", writes)

More posts simply repeat the step, and the copies accumulate in each reader's list in arrival order. The second model pushes several posts from different authors, and each feed grows by exactly the posts its owner follows. No feed receives a post from an account its owner does not follow, however many copies are flying around elsewhere. The total write count is the sum over all posts of their authors' follower counts, which is why the daily total reaches 40,000,000,000 entries at full scale.

followers = {"ana": ["ben", "cy"], "ben": ["cy"], "cy": ["ana"]}
feeds = {"ana": [], "ben": [], "cy": []}
posts = [("ana", "ana: sunny day"), ("ben", "ben: game tonight")]

for author, text in posts:
    for follower in followers[author]:
        feeds[follower].append(text)

feed = feeds["cy"]
for text in reversed(feed):
    print(text)
print("entries read:", len(feed))

The reading side of the push model is where the spending pays off. A reader opens the feed and the stored list is already there, newest first, with the entries read counted rather than assembled. Nothing is gathered, nothing is sorted, and no followed account is consulted at read time at all. Multiplied across 1,000,000,000 reads a day, that near-zero read cost is the reason the push model exists.

followers = {"ana": ["ben", "cy"], "ben": ["cy"], "cy": ["ana"]}
feeds = {"ana": [], "ben": [], "cy": []}
posts = [("ana", "ana: sunny day"), ("cy", "cy: new recipe"), ("ana", "ana: new photo"), ("ben", "ben: game tonight")]

writes = 0
for author, text in posts:
    for follower in followers[author]:
        feeds[follower].append(text)
        writes += 1

for user in sorted(feeds):
    print(user, "feed:", feeds[user])
print("feed writes:", writes)
print("ben reads:", list(reversed(feeds["ben"])))

The closing push model runs several posts through the copying step and then reads one feed back. The total writes grow with every post, while the final read only takes a ready list and reverses it into newest-first order. The wasted-copies weakness is visible in the shape even at this size: a copy is written even for a follower who never opens the feed, and the model writes it anyway because the push step cannot know who will read. At full scale that waste is the difference between the 463,000 feed writes per second the system performs and the far smaller number its readers ever see.

Fan-out on read: the pull model

Fan-out on read is the mirror strategy: no feed list is stored for a reader, and the feed is built when the reader opens it, which is why it is also called the pull model. Each post is stored once, under its author, in a post store, and copied nowhere at post time. Posting therefore writes one post and nothing else, however many followers the author has. The cost moves entirely to the read, where one read checks the posts of every followed account and then sorts them newest first.

posts = {"ana": ["ana: morning run"], "ben": [], "cy": []}
feeds = {}

posts["ana"].append("ana: new photo")

for author in sorted(posts):
    print(author, "stored posts:", len(posts[author]))
print("stored feed lists:", len(feeds))

The first pull model posts and then inspects both stores. Only the post store changes when a post is added, and the feed store beside it stays empty, because there is no feed store in this strategy at all. That emptiness is a real saving: no feed lists are stored, maintained or rebuilt for anyone. The saving is bought with future read work, and the exchange is only visible once a read actually happens.

posts = {
    "ana": [(5, "ana: morning run"), (9, "ana: new photo")],
    "ben": [(7, "ben: hello all")],
    "cy": [(4, "cy: first post")],
}
follows = {"ana": ["cy"], "ben": ["ana"], "cy": ["ana", "ben"]}

examined = 0
gathered = []
for author in follows["cy"]:
    for when, text in posts[author]:
        examined += 1
        gathered.append((when, text))

gathered.sort(reverse=True)
for when, text in gathered:
    print(text)
print("posts examined:", examined)

One read under the pull model gathers before it can show anything. The reader follows a small set of accounts, so the read examines each of those accounts' stored posts, merges them, and sorts the result newest first. The posts examined are counted in the model, and that count grows with both the number of accounts followed and how much each of them has posted. A reader who follows 200 accounts, the stated average, pays that gathering cost on every single read, with no sharing of the work between reads.

posts = {"ana": [], "ben": [], "cy": [(4, "cy: first post")]}
follows = {"ana": ["cy"], "ben": ["ana"], "cy": ["ana", "ben"]}

def feed_for(user):
    gathered = []
    for author in follows[user]:
        gathered += posts[author]
    return [text for when, text in sorted(gathered, reverse=True)]

print("first read:", feed_for("ana"))
posts["cy"].append((10, "cy: just posted"))
print("second read:", feed_for("ana"))

Freshness is the pull model's compensating strength, and the third model shows it directly. A post is added after a first read, and the second read finds the new post with no feed update in between, because there is nothing to update. Under push, the same post would wait for its copies to be written and delivered before any follower could see it. A pull feed is therefore as fresh as the post store itself, always, with no queue delay to operate or explain.

posts = {
    "ana": [(5, "ana: morning run"), (9, "ana: new photo")],
    "ben": [(7, "ben: hello all")],
    "cy": [(4, "cy: first post"), (8, "cy: market day")],
}
follows = {"ana": ["cy"], "ben": ["ana"], "cy": ["ana", "ben"]}

def read_feed(user):
    examined = 0
    gathered = []
    for author in follows[user]:
        for post in posts[author]:
            examined += 1
            gathered.append(post)
    feed = [text for when, text in sorted(gathered, reverse=True)]
    return feed, examined

for user in ["ana", "cy"]:
    feed, examined = read_feed(user)
    print(user, "feed:", feed)
    print(user, "posts examined:", examined)

The closing pull model builds two different feeds for two readers from the same stored posts. The reader who follows more accounts examines more posts to build a feed, and that reader's every read repeats the larger cost. No work is shared between the two reads, and no earlier read makes a later one cheaper, because nothing is kept from either. The model also shows the storage elegance of the strategy: one stored post serves every reader who follows its author, however many readers that turns out to be.

The verdict is fan-out on write as the default for ordinary accounts, with fan-out on read kept for the accounts the push model cannot carry. Reads arrive at 11,574 per second against posts at 2,315 per second, and the day holds 1,000,000,000 reads against 200,000,000 posts, so placing the merge at post time pays it once per post instead of once per read. Pure pull is rejected as the general strategy because it would repeat the gathering work on every one of those reads, most of them re-gathering posts that were already read many times. The cost of the verdict is admitted plainly: posting becomes the heavy operation, feed writes reach 463,000 per second under full push, and a tail of very large accounts still breaks the model — which is the next problem the design has to answer.

Deep dive: the celebrity problem

Every average hides a tail, and the follow graph's tail is the celebrity account: an account with far more followers than a normal account. The celebrity problem is the strain that one such account puts on fan-out on write, because under push, one celebrity post must be copied into every follower's feed. The scale here is stated plainly: the celebrity in the design has 50,000,000 followers. One post from that account is therefore not a bigger version of a normal post but a different kind of event entirely.

followers = 50000000
writes_per_second = 100000

writes = followers
seconds = writes // writes_per_second

print("feed writes for one post:", writes)
print("seconds to finish:", seconds)
print("minutes, about:", round(seconds / 60))

The arithmetic of one celebrity post is short and brutal. That one post makes 50,000,000 feed writes under fan-out on write, one for each follower. At 100,000 writes per second, those writes take 500 seconds, which is 500 seconds, about 8 minutes, spent spreading a single post. During those minutes the fan-out fleet is busy with one post while ordinary posts queue behind it, so the tail delays the average as well as itself.

Three ways to handle a celebrity post

A threshold is what turns the hybrid from an idea into a rule. The design treats accounts with more than 10,000 followers as huge, and that line, not fame or account type, decides the path. Accounts at or under the threshold push their posts, and accounts over the threshold are pulled at read time instead. The line is deliberately far below the celebrity's 50,000,000 followers, because the damage grows with audience long before an account reaches that size.

accounts = {"ana": 120, "ben": 350, "celebrity": 25000}
threshold = 10000

for name in sorted(accounts):
    if accounts[name] > threshold:
        plan = "pull"
    else:
        plan = "push"
    print(name, accounts[name], "followers ->", plan)

The threshold model sorts a small set of accounts into the two plans by follower count alone. Ordinary accounts, with audiences in the hundreds, are assigned to push, and the large account is assigned to pull. Nothing about the content of a post influences the decision, and an account crosses from one plan to the other only when its audience crosses the line. That mechanical quality is a virtue in operation: the rule can be applied at post time with one lookup and no judgement call.

threshold = 10000
posts = [("ana", 200), ("celebrity", 50000000)]

total = 0
for author, followers in posts:
    writes = followers if followers <= threshold else 0
    total += writes
    print(author, "post feed writes:", writes)
print("total feed writes:", total)

The hybrid's write counts show what the threshold buys in a single run. A normal post and a celebrity post arrive together, and the normal post writes one copy per follower while the celebrity post writes none at all. The total for the run is the ordinary post's count alone, as if the celebrity post were free to publish. Publishing really is nearly free for the huge account; its cost has moved to the reads of followers who actually open the feed while the post is recent.

threshold = 10000
posts = [("ana", 200), ("ben", 150), ("celebrity", 50000000)]

full = 0
hybrid = 0
for author, followers in posts:
    writes = followers if followers <= threshold else 0
    full += followers
    hybrid += writes
    print(author, "hybrid writes:", writes)

print("full push writes:", full)
print("hybrid writes:", hybrid)
print("writes saved:", full - hybrid)

The closing model runs the comparison the whole argument rests on: the same posts under full push and under the hybrid. The two ordinary posts push their copies under both strategies, and the celebrity post pushes all of its copies only under full push. Almost all feed writes in the run disappear under the hybrid, because the celebrity post makes none. The saved writes are not a rounding error or an optimisation detail but essentially the entire write load of the run, removed by one threshold rule.

The verdict is the hybrid: ordinary accounts use fan-out on write, and huge accounts are left for fan-out on read. The hybrid keeps normal posts cheap to read, because their followers still receive ready lists, and huge posts cost no copies at all, because their followers pull them at read time and merge them in. The cost is real and named: reads by followers of huge accounts do a small merge that other reads skip, and the design carries two feed paths where one would be simpler. The threshold of 10,000 followers is the admitted judgement call in the design, and accounts near the line behave acceptably under either path, so the exact placement matters far less than having a line at all.

Deep dive: ranking the feed

Ordering is a separate decision from placement, and it is made after the feed exists. Feed ranking means each post earns a score, and the feed shows the highest scores first. Chronological order, the baseline, shows posts newest first with no score at all. A score is a number built from signals about a post, and a signal is any measurable fact the order is allowed to care about.

Two signals carry the worked example, and both are defined by what they reward. Recency is the signal that gives newer posts more score than older posts, so a fresh post starts with an advantage that fades as it ages. Likes are the signal that gives well-liked posts more score, so a post other readers valued can outrank a newer post nobody valued. Combining signals is the whole craft of ranking: each signal votes, the score adds the votes, and the feed order is the score order.

Two ways to order a feed

posts = [
    ("ana: morning run", 1, 3),
    ("ben: match report", 9, 15),
    ("cy: lunch photo", 4, 5),
]

for text, hours, likes in sorted(posts, key=lambda p: p[1]):
    print(text, "| hours old:", hours)

Chronological order, shown first, sorts posts only by age. The printout lists the same small set of posts from newest to oldest, and no other fact about them is consulted. The well-liked older post sits last purely because it is oldest, which is the standing weakness the ranked order exists to fix. Keeping the baseline visible matters, because a ranked order can only be judged against the order it replaces.

posts = [
    ("ana: morning run", 1, 3),
    ("ben: match report", 9, 15),
    ("cy: lunch photo", 4, 5),
]

scored = [((10 - hours) + likes, text) for text, hours, likes in posts]
for score, text in sorted(scored, reverse=True):
    print(text, "| score:", score)

The scored model uses one stated demo rule: a post earns 10 minus its age in hours, plus its likes. Recency enters through the age term, which shrinks as the post ages, and popularity enters through the likes term, which grows as readers respond. Under that rule the well-liked older post climbs to the top, while the fresh post stays near it on recency alone. The rule is a teaching example rather than a production formula, but its shape — a fading freshness term plus an earned response term — is the shape real ranking rules share.

posts = [
    ("ana: morning run", 1, 3),
    ("ana: old photo", 6, 2),
    ("ben: match report", 9, 15),
    ("ben: hello all", 2, 0),
]

print("newest first:")
for text, hours, likes in sorted(posts, key=lambda p: p[1]):
    print(text)

print("ranked:")
scored = [((10 - hours) + likes, text) for text, hours, likes in posts]
for score, text in sorted(scored, reverse=True):
    print(text, "| score:", score)

The closing model orders one feed both ways from the same posts, so the comparison is exact rather than rhetorical. Newest first leaves the well-liked match report at the bottom, and the ranked order lifts it to first place while the newest post stays near the top. Nothing about the posts changed between the two orders; only the rule for arranging them changed. That isolation is what makes ordering a safe layer to experiment with, since a bad rule can be replaced without touching storage, fan-out or the follow graph.

The verdict is ranking as an ordering layer on top of the hybrid feed, with chronological order kept as the baseline it must beat. Ranking sits on top of either fan-out strategy, because it only changes the order of posts and never where they are stored or when they are copied. The trade-off is spoken aloud: ranking can surface better posts, and it costs scoring work on every feed, paid at read or merge time on the hot path. A service that cannot explain its scores, or whose readers value predictability over surfacing, should keep chronological order and spend the effort elsewhere.

Bottlenecks and failure modes

The first component to saturate under the hybrid is the fan-out path, and queue depth is the number that says so. Feed writes arrive for copying at an average of 463,000 per second when ordinary posting runs at the stated rates, and any burst of posting pushes arrivals above the rate the workers drain them. The queue absorbs the burst and drains late, so the visible symptom is feeds growing staler rather than requests failing. Watching that queue is the earliest honest signal that the write side is falling behind, long before readers can name the problem.

The read path is the second pressure point, and its pressure arrives as merges rather than volume. A reader who follows a huge account pays a pull and a merge on every read, and a reader who follows several huge accounts pays it several times over. Caching the pulled slice of a huge account's recent posts softens that cost, because many followers read the same recent posts in the same minutes. Ranking adds its own read-time cost on top, since scoring work is paid on every feed that ranking orders.

• Fan-out workers fall behind: posting stays fast and feeds grow staler until the queue drains; no post is lost, because the jobs wait in the queue rather than in worker memory
• The feed store loses a reader's list: the feed rebuilds by gathering the posts of the accounts that reader follows, and the reader sees a slower, pull-built feed during the rebuild rather than an error
• A huge account posts during a read burst: the post is stored once and pulled by readers, so the burst costs reads a merge step but never triggers a write storm
• The follow graph store is unavailable: posting and following pause, because neither copies nor gathers can be aimed, while already-built feed lists keep serving reads
• The ranking step fails: the feed falls back to chronological order, which needs no scores, and readers see a predictable order instead of no feed
• The cache in front of the feed store is lost: reads fall through to the feed store, slower until the cache refills, with no feed content lost

The pattern across those failures follows the stance the requirements set. Reads degrade in freshness or speed before they degrade at all, writes degrade in delay before they fail, and nothing degrades into a lost post that was confirmed to its author. The pull path doubles as the rebuild path for a lost feed list, which is a quiet benefit of keeping both strategies in the design. A component is allowed to be slow, late or rebuilt; it is not allowed to invent posts, drop confirmed posts, or show a reader another reader's feed.

Trade-offs

The design's decisions can be collected in one place, because each one was argued where it was made and none of them is free. The strategy table carried from the source design summarises the three placements of fan-out work side by side. The decisions table then records what the chosen design actually picked and what each pick gave up. Reading the two together shows the hybrid not as a compromise slogan but as a specific allocation of costs to the sides that can bear them.

The three strategies at a glance

followers = {"ana": 2, "ben": 1, "cy": 1}
posts_each = {"ana": 1, "ben": 1, "cy": 1}
posts = sum(posts_each.values())

examined = posts_each["ana"] + posts_each["ben"]
print("posts:", posts)
print("fan-out on write, feed writes:", sum(followers.values()))
print("fan-out on read, stored posts:", posts)
print("fan-out on read, one cy read examines:", examined)

The cost recap model compresses the whole comparison into one small graph where every count can be checked by eye. Each account posts once, and the printout compares the work each strategy does on exactly the same activity. Push writes one copy per follow link, pull stores only the posts themselves, and one pull read examines the posts of the accounts its reader follows. The model is tiny, but the ratios it shows are the same ratios the estimates found at full scale, which is why the small version is worth trusting as a summary.

The decisions and their costs

Taken together, the design spends write-time work to buy read-time cheapness, then refuses to spend it where the price becomes absurd. Ordinary posts pay copies so that 1,000,000,000 daily reads stay close to lookups, and huge posts pay nothing at post time so that no single author can commandeer the copying fleet. Ranking is bought separately and can be dropped separately, because ordering was never load-bearing for the storage or fan-out choices. A far smaller service, with few readers per post, should make the opposite exchange — pull everything, rank nothing — and keep the design as small as its load allows.

Common questions

How does a news feed work?

A news feed is a personal list of recent posts from the accounts a reader follows, shown newest first. Under the hybrid design, ordinary posts are copied into followers' feed lists at post time, so most of a feed is ready before it is read. Posts from huge accounts are stored once and pulled at read time, then merged into the ready list in feed order. Reading therefore stays close to a lookup plus a small merge, while posting carries the spreading work in the background.

What is fan-out in a news feed?

Fan-out is the spreading of one new post towards every follower who should see it. At the stated scale it is the largest load in the system: 2,315 posts per second, each landing in 200 feeds, make 463,000 feed writes per second. Fan-out on write performs that spreading at post time into stored lists, and fan-out on read performs the equivalent gathering at read time from posts stored once. The word names the shape of the work — one event becoming many deliveries — whichever side performs it.

How does a news feed handle a celebrity with millions of followers?

The design draws a threshold at 10,000 followers and treats accounts above it as huge. A celebrity account with 50,000,000 followers would need 50,000,000 feed writes for one post under full push, taking 500 seconds, about 8 minutes, at 100,000 writes per second. Under the hybrid that post is stored once, written nowhere at post time, and pulled by followers at read time instead. Ordinary accounts below the threshold keep pushing, so their followers' reads stay ready-made.

Should a news feed be chronological or ranked?

Chronological order shows posts newest first with no score, which is simple, predictable and free. Ranked order gives each post a score from signals such as recency and likes — in the worked example, 10 minus its age in hours, plus its likes — and shows the highest scores first. Ranking can lift a well-liked older post above weaker newer posts, and it costs scoring work on every feed. The design therefore treats ranking as a removable layer over either order's feed, never as a reason to change storage or fan-out.

Related designs reward the same habits of estimation and placement. A chat system faces the mirror problem of delivering each message to a small, named set of readers rather than to every follower, and a notification system shares the queue-and-delivery shape that fan-out uses behind the posting request. A social app design combines a news feed with photo storage, notifications and presence in one product, and an autocomplete design shows the same read-path discipline applied to keystrokes instead of posts.

The estimate at a glance