Courses › System Design › Question Breakdowns

Design Instagram

Photo storage and CDNs, read/write separation, sharding and precomputed feeds.

Design a photo-sharing service like Instagram: users upload photos, follow each other and scroll a feed of photos from the people they follow. It combines two classic problems: storing and serving large media cheaply, and generating a feed fast for hundreds of millions of users.

Requirements

Functional

  • Users can post a photo with a caption
  • Users can follow other users and see each user's profile grid
  • Users scroll a home feed of recent posts from the people they follow

Out of scope: comments, likes, stories, video, search, recommendations.

Non-functional

  • The feed opens in under ~300ms, and images start appearing right after
  • A posted photo is never lost; followers may see it a few seconds late
  • Works on poor mobile networks: uploads resume, images arrive in a size that suits the screen

[!TIP] Back-of-the-envelope 200M daily users posting 50M photos/day is ~600 uploads/second (~2k at peak). Phones shoot 2–4 MB originals, but we keep a 1080px version (~300 KB) plus three smaller renditions (~150 KB together): ~22 TB of new images a day, ~8 PB a year. Reads dwarf that: 200M users × 10 feed opens × 20 photos is 40B image fetches a day, which only a CDN can serve. Metadata is ~500 bytes per post, ~25 GB/day, so images and metadata need very different stores.

Core Entities

  • User: id, name, email, created at
  • Photo: id, owner, storage path, title, location, created at
  • Follow: follower, followee

API

POST /v1/photos            (multipart: image, title)   → { photoId }
GET  /v1/photos/{photoId}                              → metadata + CDN URL
PUT  /v1/follows/{userId}                              → 204
GET  /v1/feed?cursor=&limit=20                         → { photos[], nextCursor }
GET  /v1/users/{userId}/photos?cursor=                 → { photos[], nextCursor }

High-Level Design

Photos go to object storage behind a CDN; metadata goes to a database. Uploads and views are separate services, so slow uploads can't starve reads.

cdn = cache "CDN" @0,0
s3 = blob "Object Storage\n(photos)" @2,0
user = client "Client" @0,2
lb = service "Load Balancer" @1,2
up = service "Upload Service" @2,1
read = service "Read / Feed\nService" @2,3
meta = store "Metadata DB" @3,2
feed = cache "Feed Cache" @3,3
user -> lb
lb -> up, read
up -> s3
up -> meta
read -> meta
read -> feed
cdn -> s3 : origin
user --> cdn : image bytes

[!NOTE] Keep image bytes off your servers The Upload Service doesn't receive the photo. It returns a presigned URL, and the phone uploads straight to object storage (with multipart upload, so a dropped connection resumes instead of restarting). When storage confirms the object, the service records the metadata and queues the resize job. App servers only ever handle small JSON requests, so slow mobile uploads can't tie them up.

Deep Dives

1. Storing metadata

Start from the queries, not the entities:

Postgres sharded by user handles all four, and this is what Instagram itself ran for years: follows are stored twice (once under the follower, once under the followee) so both directions are single-shard queries. Cassandra fits the same shape just as well. Either is a fine answer if you tie it to these queries.

2. Sharding: by user or by photo?

By owner user_id *(recommended)*: the profile grid, the most common database query, hits one shard. The worry is celebrities, but their photos are read through the cache and CDN, not the database, so their shard sees little extra load.

By photo_id: spreads rows perfectly evenly, but every profile grid becomes a scatter-gather across all shards.

Use thousands of logical shards mapped onto a few physical databases. Moving a logical shard to a new machine is a copy plus a mapping change, so you never re-hash data as you grow.

3. Generating photo IDs that sort by time

Make the 64-bit ID carry both the time and the logical shard, so IDs sort by creation time and also say where the row lives. Instagram's published scheme generates them inside each shard's database:

| 41 bits: milliseconds since a custom epoch | 13 bits: logical shard | 10 bits: per-shard sequence |

41 bits of milliseconds lasts ~69 years, 13 bits address 8,192 logical shards, and 10 bits allow 1,024 IDs per millisecond per shard. No separate ID service is needed, and a feed can be sorted by ID alone.

4. Generating the feed

On read (pull): get the followees, fetch each one's latest photos, merge and rank. Correct, but with hundreds of followees it's far too slow for 200ms.

Precompute (push): when a user posts, append the photo ID to each follower's precomputed feed in the feed cache. Reading a feed is one lookup, then hydrate the photo metadata.

poster = client "Poster" @0,0
up = service "Upload Service" @1,0
q = queue "Fan-out Queue" @2,0
w = service "Fan-out Workers" @3,0
f = store "Follows" @3,1
feed = cache "Feed Cache\n(per user)" @2,1
reader = client "Follower" @0,1
read = service "Feed Service" @1,1
poster -> up
up -> q
q -> w
w -> f : who follows?
w -> feed : append photo ID
reader -> read
read -> feed

Hybrid: pushing a celebrity's photo to 50M followers is too expensive. Don't fan out for accounts above a follower threshold. Instead, merge their recent photos into the feed at read time. (The News Feed breakdown covers this in depth.)

5. Serving photos fast

  • A CDN serves photo bytes from the edge; your servers only hand out URLs.
  • Store a few resized variants (thumbnail, feed size, full) generated at upload time, so phones don't download originals.
  • Feeds mostly show posts from the last two days, so cache metadata for recent posts in Redis/Memcached: 2 days × 25 GB fits easily, and older posts fall back to the database.

6. Reliability

Acknowledge a post only after the object is stored and the metadata row is committed. Object storage keeps copies across availability zones; replicate the metadata asynchronously to a second region. A resize job that fails is retried from the queue, since the original is already safe.

What's expected at each level

  • Mid-level: blob storage + metadata DB, a working upload and feed flow, a CDN.
  • Senior: direct-to-storage uploads, sharding driven by the query list, time-sortable IDs, precomputed feeds.
  • Staff: the celebrity fan-out problem, cost of storage and egress at petabyte scale, image processing pipeline, multi-region serving.