Courses › System Design › Question Breakdowns

Design YouTube / Netflix

Upload and transcoding pipelines, adaptive streaming and CDN strategy.

Design a video platform like YouTube or Netflix: users upload videos, the platform processes them, and millions of viewers stream them on every kind of device. The two halves need very different designs: ingesting and transcoding uploads (a slow, async pipeline) and streaming at enormous scale (almost entirely a CDN problem).

Requirements

Functional

  • Creators upload videos and see their processing status
  • Viewers watch videos on phones, browsers and TVs, and resume on another device
  • Each video shows a view count

Out of scope: search, recommendations, comments, channels and subscriptions, live streaming, ads.

Non-functional

  • Playback starts in under ~2 seconds and rarely rebuffers, on good and bad networks
  • An upload that the creator was told succeeded is never lost
  • A new video may take minutes to become watchable; view counts may lag

[!TIP] Back-of-the-envelope Start from watch time: YouTube has reported over 1B hours watched a day. At an average ~2.5 Mbps that's 1B × 3,600 s × 2.5 Mb ÷ 8 ≈ 1.1 EB of video sent per day, ~100 Tbps on average. On the ingest side, 500 hours uploaded per minute is 30,000 hours of video per hour; if the full ladder of renditions costs ~1 GB per hour of video, stored output grows ~30 TB/hour. Encoding each hour of video into ~8 renditions takes on the order of 8 CPU-hours, so ~240k cores are always busy transcoding. Egress dwarfs everything, so the CDN is the design.

Core Entities

  • Video: id, uploader, title, description, status (processing / ready), duration, stats
  • Video file: one per resolution/codec, split into segments
  • Thumbnail, Comment, User, Watch progress (user, video, offset)

API

POST /v1/videos                 { title, description, tags } → { videoId, uploadUrl }
PUT  {uploadUrl}                 (resumable, chunked upload to storage)
GET  /v1/videos/{id}             → metadata + manifest URL
PUT  /v1/videos/{id}/progress    { offsetSeconds }

Creating the video returns at once with status: processing; the creator's dashboard subscribes to status changes and shows "ready" when the pipeline finishes.

High-Level Design

up = client "Uploader" @0,0
api = service "Upload Service" @1,0
raw = blob "Raw uploads" @2,0
q = queue "Processing Queue" @3,0
enc = service "Encoders" @3,1
thumb = service "Thumbnail\nGenerator" @4,0
out = blob "Video Storage\n(segments)" @3,2
meta = store "Metadata DB" @2,1
cdn = cache "CDN" @2,2
viewer = client "Viewer" @0,2
up -> api
api -> raw
raw -> q
q -> enc
q -> thumb
enc -> out
enc --> meta : ready
out -> cdn
viewer -> cdn : segments
viewer --> meta : metadata

Upload path: the client uploads the original file in resumable chunks straight to storage. A message on the processing queue then starts the pipeline: encode into multiple formats, generate thumbnails, and finally mark the video ready.

Watch path: the client fetches metadata and a manifest, then streams segments from the CDN. Your own servers are rarely on the byte path.

Deep Dives

1. Transcoding pipeline

One upload becomes dozens of outputs: several resolutions (240p–4K) × codecs (H.264, VP9/AV1) × segments. Treat it as a DAG of jobs:

  • Split the original into short chunks (a few seconds each)
  • Encode chunks in parallel across a worker fleet, one task per chunk × format
  • Generate thumbnails, extract audio, run content checks
  • Write the manifest and mark the video ready

A queue (or a workflow engine) tracks the tasks and retries failed ones. Parallel chunk encoding turns hours of encoding into minutes.

2. Adaptive bitrate streaming

Store each rendition as small segments plus a manifest (HLS or DASH). The player measures its bandwidth and picks the best rendition for the next segment, dropping to 360p in a tunnel and climbing back to 1080p on Wi-Fi without stopping. Seeking and resuming from an offset are just "start at segment N".

Resume works through the progress API: the player saves (user, video, offset) every ~10 seconds. Another device reads the offset and starts at the segment that contains it, but from its own manifest: a TV asks for 4K HEVC, an older phone for 720p H.264.

3. The CDN and the long tail

  • Tiered caching: edge servers fall back to a regional mid-tier, which falls back to origin storage. A video watched in only one region never travels to the others.
  • Caches inside ISPs: at 100 Tbps, even CDN bandwidth is expensive. Netflix (Open Connect) and YouTube (Google Global Cache) place cache servers inside ISP networks, so most bytes never cross the internet's backbone.
  • Fill ahead of demand: Netflix copies tomorrow's likely titles to those caches overnight, when links are idle.
  • Long tail: most videos are rarely watched. Cache only their first few segments at the edge, so playback starts fast while the rest streams from the mid-tier.

4. Storing metadata

Video metadata is relational and modest in size: a few KB per video, so even billions of videos are a few TB. YouTube's own answer was MySQL sharded with Vitess, which it built for this. Shard by video ID so a viral video's row lives on one shard behind a cache, and keep a small channel_videos table, sharded by channel, for the creator's video list.

One consistency case matters: a creator who just uploaded expects to see the video in their dashboard. Read the creator's own data from the primary (read-your-writes) and everyone else's from replicas.

5. Thumbnails: lots of tiny files

Every home page, sidebar and results list is a grid of thumbnails, so they're requested far more often than videos are played. They never change once made, so give each version its own URL, serve it from the CDN with a year-long cache lifetime, and let an image service produce sizes and formats (WebP, AVIF) on first request. With a ~99% CDN hit rate, the origin store hardly matters.

6. Deduplication

Two different problems hide under "dedup":

  • Identical files (the same upload sent twice): a content hash of the original catches them before transcoding, saving the compute.
  • The same content, re-encoded (cropped, sped up, filmed off a screen): this is a copyright problem, not a storage one. YouTube's Content ID compares audio and video fingerprints against reference files supplied by rights holders and lets the owner block, track or monetize the match. It runs as a step in the processing pipeline.

7. Counting views

Incrementing a counter row on every view creates a hot row for every viral video. Write view events to a stream (Kafka), aggregate them in batches, and update counts every few seconds. View counts are allowed to lag.

What's expected at each level

  • Mid-level: blob storage + metadata DB, async processing queue, a CDN for playback.
  • Senior: chunked parallel transcoding, adaptive bitrate (HLS/DASH), resumable uploads, metadata sharding and caching.
  • Staff: CDN economics (tiers, ISP caches, overnight fill), long-tail strategy, transcoding cost, fingerprinting for rights management, view counting at scale.