Courses › System Design › Key Technologies

Memcached

A simple multi-threaded cache: client-side sharding, leases, invalidation and Memcached vs Redis.

Memcached is a distributed, in-memory key-value cache — nothing more. It stores opaque byte values under string keys, evicts the least recently used items when memory fills up, and does it very fast across many threads. In interviews it's the "plain cache" alternative to Redis, and it's famous for running Facebook's caching layer at enormous scale.

How it works

  • Operations — get, set (with a TTL), delete, incr/decr, add (set only if missing), and cas (compare-and-set using a version token).
  • Multi-threaded — one server uses all its cores, so a single node handles very high throughput.
  • Slab allocator — memory is pre-divided into size classes to avoid fragmentation; each class evicts with its own LRU.
  • No persistence, no replication — a restart empties the cache, and a lost node loses its keys. That's acceptable because the database is the source of truth.
  • Client-side sharding — servers don't know about each other. The client library picks a server for each key, usually with consistent hashing, so adding or removing a node remaps only a fraction of keys.
[App server] -> (Memcached pool: hashed by key)
[App server] -> (Database: on miss, then set in cache)

Memcached vs Redis

[!TIP] Default to Redis, but know why Memcached exists In most interviews Redis is the safer answer because it covers more use cases. Memcached is a fine choice when you only need a large, simple, multi-threaded cache and want the least operational surface.

Lessons from running it at scale

These problems apply to any cache, and they're good deep-dive material:

  • Thundering herd — when a hot key expires, thousands of requests miss at once and hammer the database. Fix it with leases (only one client gets permission to refill; others wait or use the stale value) or request coalescing.
  • Stale sets — a slow reader can overwrite a newer value with an old one. Leases or cas tokens prevent it.
  • Invalidation — on a write, delete the cache key rather than updating it; deletes are idempotent and avoid races. Facebook drives deletes from the database's replication log so every region's cache is invalidated.
  • Hot keys — replicate very hot keys across several servers, or cache them in-process on the app server.

See Caching and Distributed Caching for the general patterns.

When to use it

  • A large, simple look-aside cache for database query results, rendered fragments or sessions you can afford to lose
  • Very high read throughput on multi-core machines

When not to

  • You need data structures, persistence, pub/sub, or atomic operations beyond counters — use Redis
  • The data can't be rebuilt from another source

[!INTERVIEW] Read-heavy profile service "Profiles are read 100x more than written. I'll put a Memcached pool in front of the database with cache-aside reads and a 1-hour TTL. On writes I delete the key rather than setting it, and I use leases on misses so a popular profile expiring doesn't stampede the database."