Design YouTube

Clarify the problem

Clarifying means agreeing what the service must do before designing anything.

The interview question is short. Design YouTube.

A short question hides large choices. The first job is to make those choices explicit.

Uploading is in scope. Watching is in scope.

Search by title is in scope. Recommendations are a seam only.

Live streaming is out of scope. Comments are out of scope.

What the service must do

A creator must be able to upload a video. Uploading means sending the video file from the creator's device to the service.

A viewer must be able to watch a video. Watching means the service sends the video to the viewer's device as it plays.

A viewer must be able to find a video. This design covers search by title and description.

A creator must be able to name and describe a video. That text is what search and browsing use.

What stays out

Live streaming stays out of scope. Live video arrives and plays at the same time, which is a different problem.

Comments stay out of scope. They add a social system that would hide the video problems.

Recommendations stay as a seam only. A seam is a named place where another system plugs in later.

Naming exclusions is part of the design. An interviewer can widen the scope and the design can then adapt.

How well it must work

Watching must start quickly. A viewer who waits too long leaves before the video plays.

Watching must keep working when one server fails. Playback is the core promise of the service.

An upload may take longer to appear. Viewers accept a processing delay before a new video is watchable.

That difference matters. The design can spend time on uploads to make watching fast and steady.

The shape of the problem

This design is read-heavy. Reads are views, and views vastly outnumber uploads.

Video files are large. Moving and storing them costs more than the small records around them.

Estimates put numbers on both facts. Those numbers drive every later choice.

Estimate the scale

Estimation is working out the size of a design from a few stated figures before choosing any part of it.

The scale comes first. One day holds 86,400 seconds, and daily counts divided by that figure give per-second rates.

The starting figures are stated assumptions for this design. Computed results are shown as arithmetic, step by step.

Estimation comes before components. A design sized for the wrong scale fails however neat its parts are.

Views per second

The design assumes 5,000,000,000 video views a day.

5,000,000,000 views ÷ 86,400 seconds = 57,870 views per second.

That is an average. Busy evenings run higher, and delivery must be sized above the average.

Uploads per second

The design assumes 1,000,000 uploads a day.

1,000,000 uploads ÷ 86,400 seconds = 12 uploads per second.

Compare the two rates. Views arrive thousands of times faster than uploads, so watching dominates the design.

Storage growth

An average uploaded video is 100 MB of source video. Source video is the file exactly as the creator sent it.

1,000,000 uploads × 100 MB = 100 TB of new source video a day.

Stored versions of each video total about 1.5 times the source size. Each video is stored in several versions for different devices.

100 TB × 1.5 = 150 TB of new stored video a day.

Storage only grows. Nothing in this design deletes a popular video, so capacity planning is a standing job.

Delivery volume

An average view streams 50 MB of video to the viewer.

5,000,000,000 views × 50 MB = 250 PB of video delivered a day.

Delivery dwarfs storage. A day of watching moves far more data than a day of uploading stores.

The estimates point one way. Spend the design effort on delivery first, and treat uploads as a slower background flow.

Design the upload path

Transcoding is changing a video from the form it arrived in into the forms viewers' devices can play.

A rendition is one version of a video at a set size and quality. This design targets 5 renditions for each video, a stated choice.

The upload path is a background flow. It can take minutes, because no viewer is waiting on it directly.

Receive is the first step. Chunks arrive and are checked.

Store source is the second step. The original file is kept in blob storage.

Transcode is the third step. Workers make each rendition.

Publish is the fourth step. Ready renditions become watchable.

Notify is the last step. The creator hears the video is ready.

Receiving a large file

A video file is too large for one fragile request. A dropped connection would restart the whole upload.

So the client splits the file into chunks. A chunk is a fixed-size piece of the file, sent and confirmed on its own.

In this design a chunk is 10 MB. A 100 MB video therefore arrives as chunks that can be checked one by one.

100 MB ÷ 10 MB = 10 chunks for an average video.

If a connection drops, only the missing chunks are sent again. The confirmed chunks stay stored.

Storing the source

Finished chunks are assembled into the source video. The source is stored in blob storage, which holds large files as single objects.

The service records that the source is complete. That record is the trigger for transcoding.

Keeping the source matters. If a new rendition is needed later, workers can make it from the source without asking the creator again.

The transcoding pipeline

A queue holds videos waiting for transcoding. Workers take videos from the queue and make the renditions.

Each rendition is separate work. One worker can make the small rendition while another makes the large one.

In a worked example, one rendition takes 30 minutes of worker time. That figure is a stated assumption for the example.

5 renditions × 30 minutes = 150 minutes of worker time for one video, done one after another.

Run in parallel, the same video is ready after about one rendition's time. Parallel workers buy waiting time with spare machines.

A video can publish once its first renditions are ready. The rest can follow, so a popular upload appears sooner.

Where the upload path breaks

A sudden wave of uploads fills the queue. Videos then wait longer before transcoding starts.

The design answers with priority. A small share of workers stays reserved for new uploads, and the rest clear the backlog.

A worker can fail mid-rendition. The queue hands that rendition to another worker, and the finished renditions are kept.

Idempotency protects this retry. Repeating a rendition job must produce the same stored rendition, not a duplicate video.

Why several renditions at all

One version cannot suit every viewer. A phone on a slow connection needs a small rendition, and a large screen needs a detailed one.

Serving the wrong version wastes both ways. A small rendition on a large screen looks poor, and a large rendition on a slow connection stalls.

Transcoding is the price of that fit. The design pays it once, on upload, so every later view is cheap and quick.

Deep-dive delivery

Delivery is sending a stored video to a viewer's device quickly, wherever that viewer is. The estimate put the real load on delivery.

5,000,000,000 views × 50 MB = 250 PB of video delivered a day.

No single data center should serve that alone. Distance adds delay, and one site is also one point of failure.

The player asks for a video. The nearest cache looks for the video.

On a cache hit, the video streams from the cache. On a cache miss, the cache fetches from blob storage.

The player then switches rendition as needed. Adaptation keeps playback going as conditions change.

Caches near viewers

A CDN is a network of caches placed near viewers. A cache holds copies of popular videos close to the people watching them.

The player asks the nearest cache first. A cache hit means the video streams from nearby, with a short round trip.

A cache miss means the cache fetches the video from blob storage. It then keeps a copy, so the next nearby viewer gets a hit.

Popular videos spread to caches on demand. Unpopular videos may stay in storage until someone nearby asks.

How much the caches carry

In a worked example, caches serve 95 of every 100 views. That split is a stated assumption for the example.

The origin then serves only the rest of the delivery. Origin means the blob storage and app servers behind the caches.

250 PB × 5 ÷ 100 = 12.5 PB a day still comes from the origin.

Even a strong cache leaves a large origin load. The origin must be sized for misses, not ignored.

Adaptive bitrate

Bitrate is the amount of data a stream uses each second. A higher bitrate looks better and needs a faster connection.

Adaptive bitrate is streaming that switches rendition as conditions change during playback. The player makes the switch without stopping the video.

Each rendition is stored as short segments. A segment is a few seconds of video, requested and played in order.

The player tracks its buffer. A buffer is video already fetched but not yet played, held ready on the device.

A shrinking buffer means the connection is losing. The player takes a smaller rendition for the next segments.

A growing buffer means spare capacity. The player takes a larger rendition and the picture improves.

Where delivery breaks

A new video can turn popular in minutes. Every cache misses at first, and many caches fetch the same video together.

The design answers with request collapsing. While one fetch is running, other requests for the same video wait for that fetch instead of starting their own.

A cache can also fail outright. The player then falls back to another cache or to the origin, and playback continues at a smaller rendition.

Delivery degrades gracefully by design. A worse picture is a success compared with a stopped video.

Design the data model

The video files live in blob storage, and the metadata lives apart from them.

Metadata is the small data about a video. It answers who made it, what it is called and whether it is ready to watch.

Keeping metadata separate matters. A page of search results needs many small records and no video data at all.

The metadata record

One record exists per video.

The video id is a unique name for the video. The creator id says who uploaded it.

Title and description hold text for search and browsing. The rendition list says which versions are ready.

The view count says how many views so far. The state says processing, ready or removed.

The record changes state as the upload pipeline works. A video in the processing state is stored but not yet watchable.

Search reads this store constantly. An index on title text lets search find videos by words in the title or description.

New records arrive at the upload rate. 1,000,000 uploads ÷ 86,400 seconds = 12 new metadata records per second.

That write rate is gentle. The hard metadata problem is not creating records, but updating one field on the most popular ones.

The hot view counter

Every view updates a view count. Views arrive at 57,870 per second across the service, and a hit video takes a large share of them.

One counter row updated by every view becomes a hot spot. Requests queue up to change the same value.

So view events are collected into batches and applied together. A batch is a group of events held briefly before it is written.

In this design, one batch holds 1,000 view events. That batching choice is a stated assumption for the worked figure.

57,870 views per second ÷ 1,000 events per batch = about 58 counter writes per second.

The price is freshness, because a displayed count can lag the true count by the batching delay. View events are also time-series data, and kept as a series they show a creator how watching rises and falls.

The recommendation seam

A recommendation is a suggested next video shown beside or after the current one. This design names where recommendations plug in and does not design the ranking itself.

The seam works like this. The metadata service supplies candidate videos and their records, and a separate ranking system orders them for the viewer.

The watch page asks the seam for a list and renders whatever comes back. If the ranking system is down, the page falls back to popular videos from the same category.

That fallback is graceful degradation. The page stays useful, just less personal, while a part of the system is failing.

Where metadata breaks

A metadata store failure must not stop playback. Players read rendition locations from caches, so watching continues while records are repaired.

A removed video must stop everywhere. Its record flips to the removed state, caches are told to drop it, and search stops returning it.

Metadata is small but central. The design replicates it widely and keeps the video path able to run from cached copies.

Evaluate the design

Evaluation means judging the finished design against the ask, honestly. A trade-off is a choice that gains one quality by giving up another.

The ask was a read-heavy video service. The design spends most of its machinery on watching, and that matches the estimates.

5,000,000,000 views × 50 MB = 250 PB of video delivered a day.

Against that, stored video grows by about 150 TB a day. Delivery is the giant, and the design treats it as one.

Many renditions gain fit for every device. They give up storage and transcoding time.

Cache-first delivery gains speed and origin relief. Removal spreads slowly.

Batched view counts gain a calm counter. Counts lag behind.

Publishing early renditions gains faster appearance. First viewers get lower quality.

Renditions against storage

Five renditions fit one video to many devices and connections. They also multiply storage and transcoding work for every upload.

The design accepts that cost on the write side. Writes are rare here, and reads are constant, so paying once to help every read is the right side.

A change could soften it. A video nobody watches could keep fewer renditions, and a rising video could gain the rest on demand.

Freshness against calm counters

Batched view counts keep a hot counter calm. They give up exact, instant counts.

That trade suits this service. A viewer choosing a video does not need the count to be right to the last view.

A different service might choose otherwise. Where a count decides money or access, exactness would win and the design would shard the counter instead.

Speed against control

Cache-first delivery is fast and cheap. It gives up instant control, because copies live in many places at once.

A removal therefore takes time to spread. The design accepts minutes of spread and makes the metadata record the source of truth for takedowns.

Availability was favoured over strict consistency here. A slightly stale list is a small harm, and a stopped video is a large one.

What would change the design

Live streaming would change it most. Live video cannot be transcoded at leisure or cached ahead of viewing.

A far smaller service would change it too. At low view rates, one storage system and no cache fleet would be simpler and cheaper.

Estimate first, pay on the rare side, and let delivery degrade in quality rather than stop.

The estimates in one place