Design a Recommendation System

The ask

The design serves a shopping, media or social service.

Each request asks for a short list for one person, served at once.

The first questions fix the surface. A home page, a product page and an email each need a different list.

The next questions fix what a good pick means. A good pick may end in a purchase, a read, a listen or a save.

Popularity alone is not the target. A list that fits the person beats a list that fits everyone.

Estimate the scale

The service takes 1,000,000,000 recommendation requests a day.

1,000,000,000 requests ÷ 86,400 seconds = 11,574 requests per second.

The catalogue holds 100,000,000 items.

Candidate generation is the fast first stage that picks a small set of items that might fit.

Ranking is the slower second stage that orders the candidates by predicted value for the person. Candidate generation leaves 500 candidates, and the screen shows the top 20.

100,000,000 items ÷ 500 candidates = 200,000. The first stage cuts the catalogue by that factor.

Data and features

Every view, click, purchase, save and skip becomes an event in a log.

Each event carries the person, the item, the time and the context. Context means facts like the device and the place.

Features turn events into measured facts for the model. Recent categories viewed and recent item popularity are two such facts.

A label comes from what the person did next. A purchase is a strong positive label, and a fast skip is a weak negative one.

Item records carry category, price, language and availability. An item that cannot be delivered should leave the pool before ranking.

High-level design

A request reaches the service through an app server, carrying the person and the context.

Candidate sources run side by side. One follows recent viewing, one follows similar people, and one follows similar items.

Hard filters remove unavailable items, owned items and items the person has blocked.

The ranking model scores the survivors with features about the person, the item and the pair.

Item embeddings sit in a vector index. Nearest neighbour search over that index supplies the similar-items source.

One embedding per item takes 4 KB, or 4,000 bytes. 100,000,000 items × 4,000 bytes = 400,000,000,000 bytes of item vectors, or 400 GB.

Training the model

Training data comes from logged requests and what people did with the lists shown.

Items shown and skipped give negative examples. Items never shown give no evidence either way.

Training only on clicked items teaches a narrow view. The log keeps shown items, clicked or not, for that reason.

The feature store feeds the same feature values to training and to serving. A mismatch between the two is training-serving skew.

The model retrains on a regular schedule as new events land. Fast-moving catalogues need shorter gaps between runs.

Serving predictions

11,574 requests × 500 candidates = 5,787,000 ranking scores per second.

The ranker scores its 500 candidates in about 50 milliseconds.

Candidate lists are precomputed per person and per topic, then refreshed as events arrive. Serving reads them instead of scanning the catalogue.

Each stage gets a deadline. A slow candidate source is dropped, and ranking runs on what arrived in time.

A fallback list covers failures. A popular list for the person's segment beats an empty screen.

Deep dive: the cold start

Cold start is the problem of recommending for a person or an item with no history yet.

A new person starts with a few sign-up choices and broad popular picks. The first clicks and saves become real features fast.

A new item starts from its category, price and words. Similar known items lend it a starting embedding.

A few slots show uncertain items on purpose. Their outcomes show whether the items deserve wider showing.

Early picks stay safe and easy to correct. One bad first list can end the relationship at once.

Deep dive: event recommendations

An event is an item with a fixed time and a fixed place. Concerts, classes and local matches are the shape.

The same two stages carry the design. Candidate generation gathers possible events, and ranking orders them.

Time and place act as hard constraints on candidates. An event that has ended, or sits too far away, leaves before ranking.

Candidate sources lean on nearness, followed organisers and past attendances.

Ranking weighs start time, travel time and free hours in the person's day. A high score cannot repair a clash with an existing plan.

Seats run out, so availability is checked again at serving time. A sold-out event is never shown as open.

Evaluation and trade-offs

Offline tests replay past requests and grade the order against known outcomes. Orders that put purchased items higher score better.

Offline tests stay blind to items that were never shown. A strong offline score still earns only a small live test first.

An online experiment gives the new model a small share of live requests. Purchases, saves, hides and returns are compared with the old model.

Complaints and fast skips act as stop signals during rollout. A rise in either one can halt the new model.

The standing trade-off is sameness against fit. A list of near-identical items can score well and still bore the person, so variety is a serving rule.

The funnel in numbers