How Things Work

Caching Strategies

How a layer of memory turns 80ms database reads into sub-millisecond hits — cache-aside vs write-through, TTLs, LRU eviction, and the hard problem of keeping cached data fresh.

How It Works

A cache is a fast, in-memory copy of data that's expensive to recompute or fetch. By serving repeated reads from memory instead of the database, caching slashes latency and offloads the origin — at the cost of a second copy of the truth that can go stale. Getting the read pattern, eviction, and invalidation right is what separates a fast system from a buggy one.

Read-Through the Cache First

Every read checks the cache before the database. A HIT returns in microseconds from memory. A MISS falls through to the slower source of truth — so the goal is to make hits the common case (a high hit ratio).

Populate on Miss (Cache-Aside)

On a miss the app fetches from the database, then writes the result into the cache with a TTL. The next request for that key is a hit. This 'lazy loading' means only data that's actually requested gets cached.

Eviction When Full

Cache memory is finite, so when it fills, an eviction policy decides what to drop. LRU (Least Recently Used) is the default — it assumes recently accessed keys will be accessed again. LFU, FIFO, and random are alternatives for different access patterns.

Keeping It Fresh

Cached data can drift from the source. TTLs bound staleness automatically. On writes you either invalidate the key (delete it, repopulate on next read) or update it (write-through). The hard part — 'cache invalidation' — is choosing the strategy that fits your consistency needs.

Key Concepts

🎯Cache Hit / Miss

A hit is found in cache (fast); a miss falls through to the origin (slow). Hit ratio = hits / total reads — the headline metric.

📥Cache-Aside

The app reads the cache, and on a miss loads from the DB and populates the cache itself. The most common pattern.

✍️Write-Through

Writes go to the cache AND the database synchronously, keeping them consistent at the cost of write latency.

⏳Write-Back

Writes hit the cache and are flushed to the DB asynchronously — fast, but risks data loss if the cache dies before flushing.

⏰TTL

Time-To-Live: an expiry on each entry that bounds how stale cached data can become without manual invalidation.

🗑️LRU Eviction

Least Recently Used — when full, drop the entry that hasn't been accessed for the longest. Good default for temporal locality.

Cache-aside with Redis

tsx

1// Cache-aside (lazy loading) — the most common pattern.
2// The application, not the cache, owns the read-through logic.
3
4async function getUser(id) {
5  const key = `user:${id}`;
6
7  // 1. Try the cache first
8  const cached = await redis.get(key);
9  if (cached) return JSON.parse(cached);     // HIT
10
11  // 2. MISS → read the source of truth
12  const user = await db.users.findById(id);
13
14  // 3. Populate the cache with a TTL so it can't go stale forever
15  await redis.set(key, JSON.stringify(user), "EX", 300); // 5 min
16  return user;
17}
18
19// On write, INVALIDATE rather than trust the old value:
20async function updateUser(id, patch) {
21  const user = await db.users.update(id, patch);
22  await redis.del(`user:${id}`);   // next read repopulates
23  return user;
24}

💡

Why This Matters

Caching is the highest-leverage performance optimization in most systems — a good hit ratio can cut database load by 90%+. But it's also where subtle correctness bugs (stale data) and incidents (cache stampedes, hot keys) hide. 'There are only two hard things in CS: cache invalidation and naming things' is a cliché for a reason.

Common Pitfalls

⚠No invalidation on writes — users see stale data because the cache still holds the old value.

⚠Cache stampede: a hot key expires and thousands of requests miss at once, overwhelming the origin.

⚠Caching everything, including rarely-read data — wastes memory and lowers the effective hit ratio.

⚠Ignoring eviction policy — the wrong policy (or none) thrashes the cache for your access pattern.

⚠Write-back caches that lose un-flushed writes when the cache node crashes.

Real-World Use Cases

1The Thundering Herd on Cache Expiry

Scenario

A popular product page is cached with a 60s TTL. When it expires, thousands of concurrent requests all miss at once and hammer the database simultaneously, spiking load and latency.

Problem

A naive TTL means every client rediscovers the miss at the same instant. The single hot key turns into a stampede against the origin — the 'thundering herd' or 'cache stampede'.

Solution

Add request coalescing (a mutex/single-flight so only the first miss recomputes while others wait) and stagger expirations with a small random jitter on the TTL. For critical keys, refresh ahead of expiry in the background.

💡

Takeaway: Hot keys need stampede protection, not just a TTL. Single-flight + TTL jitter prevents a synchronized rush of misses from overwhelming the origin.

2Stale Prices After an Update

Scenario

An admin changes a product's price, but customers keep seeing the old price for minutes because it's still cached.

Problem

The cache was populated read-through with a long TTL and nothing invalidated it on write. The cached value silently diverged from the database — a correctness bug, not just a performance one.

Solution

On every write, explicitly delete (invalidate) the affected cache key so the next read repopulates fresh data. For data that must always be correct, prefer write-through so the cache updates atomically with the database.

💡

Takeaway: Caching introduces a second copy of the truth. Decide your invalidation strategy up front — TTL bounds staleness, but writes that must be seen immediately require explicit invalidation or write-through.

Join the discussion

Comments

Loading comments...

Cookie preferences

We use essential cookies for sign-in and preferences. With your permission, we also use basic analytics to improve lessons.