Caching

Where to cache, cache-aside vs write-through, eviction, invalidation and stampedes.

Caching keeps frequently used data somewhere faster than its source of truth. It's the most common answer to "how do we make reads faster?" — and the most common source of subtle bugs. Interviewers want to see that you know both.

Why cache?

Memory is roughly 100× faster than an SSD-backed database query, and a cache absorbs load that would otherwise hit your database. A cache makes sense when:

Reads significantly outnumber writes
The same data is read repeatedly (a skewed, "hot" working set)
Slightly stale data is acceptable — or you have a clear invalidation strategy

Where to cache

Client -> CDN -> [App Server (in-process cache)] -> (Redis) -> (Database)

Caching strategies

Cache-aside (lazy loading) — the default. The app checks the cache; on a miss, it reads the DB and populates the cache.

def get_product(product_id):
    key = f"product:{product_id}"
    cached = redis.get(key)
    if cached is not None:
        return cached                                   # hit
    product = db.fetch_one("SELECT * FROM products WHERE id = %s", product_id)  # miss
    redis.set(key, product, ex=300)                     # keep it for 5 minutes
    return product

Write-through — writes go to the cache and the DB together. The cache is always fresh, but every write pays the cost, and you cache data that may never be read.

Write-behind (write-back) — writes go to the cache, which flushes to the DB asynchronously. Very fast writes, but you can lose data if the cache fails before flushing.

Read-through — like cache-aside, but the cache library loads from the DB itself.

Eviction policies

Caches are finite. When full, something gets evicted:

LRU (least recently used) — the sensible default
LFU (least frequently used) — better when popularity is stable
TTL — entries expire after a fixed time, which bounds staleness

Invalidation — the hard part

When the source data changes, the cache must change too. Options, from simplest to strongest:

TTL only — accept up to N seconds of staleness. Often good enough.
Delete on write — after updating the DB, delete the cache key so the next read reloads it.
Event-driven invalidation — publish change events (e.g. via change data capture from the write-ahead log) that invalidate or update caches.

[!WARNING] The delete-then-race problem If a reader misses the cache, reads the old value from the DB, and writes it to the cache *after* a writer deletes the key, the stale value lives on until its TTL. Short TTLs bound the damage; versioned values prevent it.

Failure modes to mention

Cache stampede (thundering herd): a hot key expires and thousands of requests hit the DB at once. Fix with request coalescing (only one request recomputes), a lock, or early probabilistic refresh.
Hot keys: one key gets so much traffic that its cache node is overloaded. Replicate the key across nodes or add a small in-process cache in front.
Cold start: after a deploy or cache failure, everything misses. Warm the cache, or make sure the DB can survive the burst.
Cache down: decide whether to fail open (go to the DB, possibly with rate limiting) or degrade the feature.

For sharding a cache across many nodes, hot-key replication and consistency at scale, see Caching at Scale.

[!INTERVIEW] Always say what you're caching and why "I'll cache the event details and seat map in Redis with a 30-second TTL — they're read millions of times during a sale and a few seconds of staleness is fine, since the booking path checks the database anyway."