← All articles

Backend & Data

Caching and the two hard problems

2026-07-15 · 5 min read

There's a famous joke: there are only two hard things in computer science — cache invalidation, naming things, and off-by-one errors. Caching itself is easy and gives you enormous speed almost for free: remember the answer to an expensive question, and next time hand back the copy instead of recomputing. The hard part is never the caching. It's knowing when your copy has gone stale.

How a cache earns its keep

Cache hit versus miss Request Cache copy + TTL clock Origin slow / expensive request HIT → return now (fast) MISS → fetch, then store
A hit returns the stored copy instantly. A miss pays the full cost once, then caches the result so the next request is a hit.

Put a cache between something slow (a database query, an API call, a rendered page) and the thing asking for it. The first request is a miss: it pays full price and stores the answer. Every request after is a hit, served from the copy at a fraction of the cost. It's the same idea a CDN uses globally, and the fix for a slow endpoint that recomputes the same thing all day.

The hard part: staleness

The moment you keep a copy, you've created the possibility of it being wrong — the underlying data changed but the cache still hands out the old value. That's the two-edged sword: a cache trades freshness for speed. Serve a price that changed five minutes ago and you've got a bug that no test catches, because the code is correct — the copy is just old.

The gist

Caching is easy: keep a copy, serve it fast. Invalidation is hard: knowing the exact moment that copy became a lie, and throwing it away in time.

Ways to keep copies honest

  • TTL (time-to-live). Let each entry expire after N seconds. Simple and self-healing — you just accept data can be up to N seconds stale. The default for good reason.
  • Explicit invalidation. When the data changes, actively delete or update the cached entry. Precise, but you have to catch every path that changes the data — miss one and you serve stale forever.
  • Versioned keys / cache-busting. Change the key when the content changes (style.css?v=2) so old and new never collide — the trick browsers force on you with favicons.

The engineering judgement is picking how much staleness each thing can tolerate: a stock price, seconds; a blog post, minutes; a company logo, forever-until-you-say-so. Match the strategy to the tolerance and caching stays the cheap win it should be — instead of the source of your weirdest bug.

CachingPerformanceBackendInvalidation
← Back to the blog