Blob Storage

Object storage, presigned URLs, multipart uploads, and keeping metadata in the database.

Images, videos, PDFs, backups, model weights: large, unstructured files don't belong in your database. They go in blob storage (also called object storage) such as Amazon S3, Google Cloud Storage or Azure Blob Storage. It's cheap, effectively unlimited, extremely durable, and in interviews it's the default home for any file bigger than a few KB.

How object storage works

Files are objects in a flat namespace of buckets and keys: s3://user-uploads/avatars/42.jpg. The slashes are just part of the name, not real folders.
You read and write whole objects over HTTP (PUT, GET, DELETE). There's no editing in place: changing an object means writing a new version.
Each object carries a little metadata (content type, size, custom tags), but you can't query across objects. Anything you need to search or join lives in your database.
Durability is designed for eleven nines (99.999999999%) by replicating or erasure-coding data across several data centers.
Modern S3 gives strong read-after-write consistency: once a PUT succeeds, every read sees it.

The standard pattern: metadata in the DB, bytes in the bucket

Store the file in blob storage and a row about it in your database: owner, key, size, status, created time. The database answers "which photos does user 42 have?"; blob storage serves the bytes.

user = client "Client" @0,1
api = service "API Server" @1,0
db = store "Metadata DB" @2,0
s3 = blob "Blob Storage" @1,2
cdn = service "CDN" @2,2
user -> api : 1. request upload URL
api -> db : 2. row (status: pending)
api --> user : 3. presigned URL
user -> s3 : 4. PUT file directly
s3 --> api : 5. upload event
cdn -> s3 : origin fetch
user -> cdn : download

Presigned URLs

Don't stream large uploads through your API servers: it ties up their bandwidth and memory for no reason. Instead, the server issues a presigned URL: a short-lived link, signed with the server's credentials, that lets the client upload (or download) one specific object directly. Your servers handle only small JSON requests, and the storage service handles the heavy bytes.

Afterwards, confirm the upload, either by having the client call back or (more reliably) by listening to the storage service's event notification, and mark the row uploaded.

Large files

Multipart upload: split a big file into parts (for example 5–100 MB each), upload them in parallel, and retry only the parts that fail. S3 assembles them at the end. Essential for video.
Range reads: download just a byte range, which lets video players seek and lets downloads resume.
Chunking with content hashes: sync products such as Dropbox split files into chunks named by their hash, so an edit re-uploads only changed chunks and identical chunks are stored once.

See Handling Large Blobs for the full pattern.

Serving files

Put a CDN in front of the bucket for anything read often or globally. The bucket is the origin; the CDN caches copies close to users. For private files, serve through signed CDN URLs or presigned GETs with short expiry.

Cost and lifecycle

Storage is cheap per GB but adds up at petabyte scale, and data transfer out to the internet is often the bigger bill. Use storage classes: hot data in the standard tier, older data moved automatically by lifecycle rules to infrequent-access and then archive tiers (Glacier-style, minutes to hours to retrieve). Delete what you no longer need — abandoned multipart uploads included.

[!INTERVIEW] The one-liner "Clients upload directly to S3 with a presigned URL; we store the object key and metadata in Postgres, mark the upload complete from the S3 event, and serve downloads through CloudFront" handles bandwidth, durability, metadata and latency in one breath.