A cache keeps a copy of something that was expensive to get, so the next request is fast. Caches sit everywhere between a user and your database: in the browser, on a CDN, in a reverse proxy, in your process, in Redis, and inside the database itself. This post shows what each layer is for, how HTTP cache headers really work (with curl demos against a tiny local server), the standard application patterns, how to invalidate safely, and how to stop a cache stampede. It ends with a Redis cache-aside example in Python.

Series map: 01 Text Encoding → … → 12 OWASP Top 10 → 13 Caching (this post) → 14 API Reliability → … → 24 Desk Health. Part C · Building safely.

Learning Outcome

  • Explain what a cache buys you and what it costs: staleness, memory, complexity.
  • Name the cache layers from browser to database, and who controls each one.
  • Write correct Cache-Control headers (RFC 9111): max-age, s-maxage, no-cache vs no-store, private, public, immutable, stale-while-revalidate.
  • Use ETag / If-None-Match and Last-Modified to get cheap 304 Not Modified answers, and check them with curl -I.
  • Bust caches with content-hashed file names.
  • Use functools.lru_cache, and implement cache-aside with Redis or Valkey, plus TTL jitter.
  • Prevent a cache stampede with locking, request coalescing and early refresh.

Chapter 1 — Why cache, and what it costs

Without a cache, every request pays the full price: a network round trip, a database query, a template render. With a cache, most requests pay a lookup instead. The share of requests answered from the cache is the hit ratio (hits / (hits + misses)). A cache with a 95% hit ratio sends only 1 request in 20 to the slow system behind it.

Caching is never free:

Cost What goes wrong
Staleness The copy is older than the truth. A user changes a price and still sees the old one.
Memory Every copy takes RAM or disk. Full caches must evict something.
Complexity New failure modes: invalidation bugs, stampedes, one user's page served to another.
Debugging "It works on my machine" becomes "it works after a hard refresh".

A good rule: add a cache when you have measured a slow, repeated read, and you can say how stale the data is allowed to be.

Terminology

  • Hit / miss — the value was in the cache / it had to be fetched from the source (the origin).
  • TTL (time to live) — how long an entry counts as fresh.
  • Fresh / stale — younger / older than its TTL. Stale entries may still be used in some cases.
  • Eviction — removing entries to make room, often LRU (least recently used first).
  • Invalidation — removing or updating an entry because the source changed.
  • Cache key — what identifies an entry: a URL, or a string like product:42. A cold cache is empty (just started); a warm one is full of useful entries.

Chapter 2 — Where caches live

A request can pass many caches on its way to the data. Each one is closer to the user (faster) or closer to the data (fresher).

Layer Who shares it Who controls it How to invalidate
Browser cache One user Your HTTP headers You cannot reach in; use short TTLs or new URLs
CDN edge Everyone in a region Headers + CDN settings Purge API, new URLs
Reverse proxy (nginx, Varnish) All users of your site Headers + proxy config Purge, restart, new URLs
In-process memory (lru_cache, a dict) One process Your code cache_clear(), restart
Shared cache (Redis, Valkey, Memcached) All app servers Your code DEL, TTL, key versioning
Database buffer cache All queries The database Automatic
OS page cache All programs The kernel Automatic

The database layer works on its own. MongoDB's WiredTiger engine, for example, keeps an internal cache that by default is the larger of 50% of (RAM − 1 GB) or 256 MB, and also relies on the operating system's file cache. MongoDB Course 15 · Performance Deep explains the working set and WiredTiger cache in detail. The rest of this post is about the layers you control.

Chapter 3 — HTTP caching with Cache-Control (RFC 9111)

HTTP caching is defined by RFC 9111 "HTTP Caching" (June 2022). It separates two ideas:

  • Freshness — "may I use my copy without asking?" controlled mainly by Cache-Control: max-age.
  • Validation — "my copy is stale; is it still good?" done with ETag or Last-Modified (next chapter).

It also separates two kinds of cache. A private cache belongs to one user (the browser). A shared cache serves many users (a CDN or proxy).

The response directives you will actually use:

Directive Meaning
max-age=N Fresh for N seconds. RFC 9111: "The response is to be considered stale after its age is greater than the specified number of seconds."
s-maxage=N Like max-age, but only for shared caches, and it overrides max-age there.
no-cache May be stored, but must be revalidated with the origin before every use. It does not mean "do not cache".
no-store Must not be stored at all, by any cache. Use it for sensitive data.
private Only the user's own browser may store it; shared caches must not.
public Any cache may store it, even when it normally would not (for example, a response to a request with Authorization).
must-revalidate Once stale, never use it without a successful revalidation.
immutable (RFC 8246) The body will never change while fresh, so do not even revalidate on reload.
stale-while-revalidate=N (RFC 5861) After expiry, a cache may keep serving the stale copy for up to N more seconds while it refetches in the background.
stale-if-error=N (RFC 5861) If the origin fails, a stale copy may be served for up to N seconds.

Other headers you will meet:

  • Age: 42 — added by shared caches: how many seconds the copy has been in a cache.
  • Expires: <date> — the old way to say "fresh until". If max-age is present, it wins.
  • No Cache-Control and no Expires, but a Last-Modified? Caches may guess a lifetime. RFC 9111 says a typical guess is 10% of the time since the last change. This is a classic source of "why is this page stale?". Always send explicit headers.

Common pitfall: using no-cache for a bank statement. The browser still writes it to disk; it just asks before showing it. For sensitive pages use no-store (and usually private).

Chapter 4 — Validation: ETag, Last-Modified and 304

When a cached copy is stale (or marked no-cache), the cache sends a conditional request:

  1. The first response carried a validator: ETag: "b5099f092335ab78" (a fingerprint of the body) and/or Last-Modified: <date>.
  2. Later, the client asks If-None-Match: "b5099f092335ab78" (or If-Modified-Since: <date>).
  3. If nothing changed, the server answers 304 Not Modified with no body. The client reuses its copy. Only headers cross the network.

An ETag starting with W/ is a weak validator: "the same meaning", not byte-for-byte equal. If both If-None-Match and If-Modified-Since arrive, If-None-Match wins (RFC 9110). ETags are more precise, because dates only have one-second resolution.

Vary tells caches which request headers change the response. With Vary: Accept-Encoding, the gzip and brotli versions are cached separately. Be careful with Vary: Cookie or Vary: User-Agent: they split the cache into so many versions that almost nothing hits.

A local server to experiment with

This standard-library server serves four paths, each with a different policy, and answers conditional requests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# cache_server.py — four caching policies and ETag validation, standard library only
import hashlib
import time
from email.utils import formatdate, parsedate_to_datetime
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 
STARTED = time.time()
PAGES = {  # path: (body, Cache-Control)
    "/": (b"<h1>Home</h1>\n", "no-cache"),
    "/static/app.3f9a2c1b.js": (b"console.log('v1');\n", "public, max-age=31536000, immutable"),
    "/news": (b"<p>Latest news</p>\n", "public, max-age=60, stale-while-revalidate=30"),
    "/api/me": (b'{"user": "ada"}\n', "private, no-store"),
}
 
class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.respond(send_body=True)
 
    def do_HEAD(self):                      # curl -I sends HEAD
        self.respond(send_body=False)
 
    def respond(self, send_body):
        if self.path not in PAGES:
            self.send_error(404)
            return
        body, policy = PAGES[self.path]
        etag = '"' + hashlib.sha256(body).hexdigest()[:16] + '"'
        if self.not_modified(etag):
            self.send_response(304)         # "your copy is still good", no body
            self.send_header("ETag", etag)
            self.send_header("Cache-Control", policy)
            self.end_headers()
            return
        self.send_response(200)
        self.send_header("Content-Length", str(len(body)))
        self.send_header("Cache-Control", policy)
        self.send_header("ETag", etag)
        self.send_header("Last-Modified", formatdate(STARTED, usegmt=True))
        self.end_headers()
        if send_body:
            self.wfile.write(body)
 
    def not_modified(self, etag):
        inm = self.headers.get("If-None-Match")
        if inm is not None:                 # If-None-Match wins over If-Modified-Since
            tags = [t.strip() for t in inm.split(",")]
            return "*" in tags or etag in tags or f"W/{etag}" in tags
        ims = self.headers.get("If-Modified-Since")
        if ims:
            try:
                return int(STARTED) <= parsedate_to_datetime(ims).timestamp()
            except (TypeError, ValueError):
                return False
        return False
 
if __name__ == "__main__":
    print("Serving on http://localhost:8000")
    ThreadingHTTPServer(("127.0.0.1", 8000), Handler).serve_forever()

Run python cache_server.py, then in a second terminal:

1
2
3
4
5
6
7
curl -I localhost:8000/                       # 200, Cache-Control: no-cache, ETag: "b5099f092335ab78"
ETAG=$(curl -sI localhost:8000/ | grep -i '^etag' | cut -d' ' -f2 | tr -d '\r')
curl -s -o /dev/null -w "%{http_code}\n" -H "If-None-Match: $ETAG" localhost:8000/        # 304
curl -s -o /dev/null -w "%{http_code}\n" -H 'If-None-Match: "old"' localhost:8000/        # 200
curl -sI localhost:8000/static/app.3f9a2c1b.js | grep -i cache-control   # max-age=31536000, immutable
curl -sI localhost:8000/api/me | grep -i cache-control                   # private, no-store
 
1
2
3
4
5
curl.exe -I http://localhost:8000/
$etag = (curl.exe -sI http://localhost:8000/ | Select-String '^ETag:').ToString().Split(' ')[1]
curl.exe -s -o NUL -w "%{http_code}`n" -H ('If-None-Match: ' + $etag.Replace('"', '\"')) http://localhost:8000/   # 304
(Invoke-WebRequest -UseBasicParsing -Method Head http://localhost:8000/news).Headers['Cache-Control']

Watch out: Windows PowerShell 5.1 drops embedded double quotes when it passes arguments to a native program such as curl.exe. Without the .Replace('"', '\"'), the server receives If-None-Match: b5099f092335ab78 (no quotes), the tags do not match, and you get 200 instead of 304. Also, in 5.1 Invoke-WebRequest throws an error on a 304 response, so use curl.exe for conditional requests.

In a browser, open DevTools → Network, load http://localhost:8000/ twice, and look for the 304 on the second load.

How a cache treats four Cache-Control policies as its copy ages, and a conditional request that returns 304 Not Modified or 200 with a new ETag

Chapter 5 — Recipes, CDNs and cache busting

Pick a policy per kind of content

Content Cache-Control Why
Hashed static files (app.3f9a2c1b.js, fonts, images) public, max-age=31536000, immutable The name changes when the content changes, so the old copy can live forever.
HTML pages no-cache (plus an ETag) Always check, usually get a cheap 304, and pick up new asset names at once.
Public data that may lag a little (news, prices list) public, max-age=60, stale-while-revalidate=30 Fast for everyone, at most about 90 s old.
Per-user API responses private, no-cache or private, no-store Must never land in a shared cache.
Secrets (statements, tokens, admin pages) no-store Never written anywhere.

CDNs and shared caches

A CDN is a shared cache in many cities. Three extra points:

  • Use s-maxage to give the CDN a longer life than browsers: Cache-Control: public, max-age=60, s-maxage=3600. RFC 9213 also defines CDN-Cache-Control, a header that only CDNs read.
  • Most CDNs offer a purge API, and many can purge by tag. The details (and support for stale-while-revalidate) are vendor-specific, so check your CDN's docs.
  • Never let a shared cache store personal data. If /account is cached publicly, the next visitor sees the previous user's page. That is a Broken Access Control bug (see Dev Essentials 12 · OWASP Top 10). Mark per-user responses private, and be careful with any response that sets a cookie.

Cache busting with hashed file names

The long-lived policy above only works if each new version of a file gets a new URL. Build tools do this by putting a hash of the content in the file name. Next.js is a common example. Its docs say that it sets "public, max-age=31536000, immutable" for truly immutable assets, that these files "contain a SHA-hash in the file name", and that this header "cannot be overridden". The HTML then points to the new names, and the HTML itself is revalidated on each visit.

The idea in a few lines of Python:

1
2
3
4
5
6
7
8
9
# fingerprint.py — content-hashed file names for cache busting
import hashlib, json, pathlib, shutil
 
src = pathlib.Path("static/app.js")
digest = hashlib.sha256(src.read_bytes()).hexdigest()[:8]
dest = src.with_name(f"{src.stem}.{digest}{src.suffix}")     # static/app.5e39b3ff.js
shutil.copyfile(src, dest)
pathlib.Path("static/manifest.json").write_text(json.dumps({"app.js": dest.name}))

Templates look up app.js in manifest.json and write the hashed name into the page. Change one byte of app.js and the name changes, so every cache fetches it fresh. A hash in the path beats a manual ?v=42 query string: it never needs a version bump, and some caches ignore query strings.

Chapter 6 — In-process caching with functools

The fastest cache is a dictionary in your own process. Python ships two ready-made ones:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# memo.py — functools caches
import time
from functools import cache, lru_cache
 
@lru_cache(maxsize=128)                 # keeps the 128 most recently used results
def exchange_rate(currency: str) -> float:
    time.sleep(0.5)                     # pretend this is a slow API call
    return {"EUR": 0.92, "JPY": 151.3}.get(currency, 1.0)
 
@cache                                  # Python 3.9+; same as lru_cache(maxsize=None)
def fib(n: int) -> int:
    return n if n < 2 else fib(n - 1) + fib(n - 2)
 
t = time.perf_counter(); exchange_rate("EUR"); print(f"miss {time.perf_counter() - t:.3f}s")  # 0.500s
t = time.perf_counter(); exchange_rate("EUR"); print(f"hit  {time.perf_counter() - t:.6f}s")  # 0.000001s
print(exchange_rate.cache_info())       # CacheInfo(hits=1, misses=1, maxsize=128, currsize=1)
print(fib(200))                         # instant; without @cache this would never finish
exchange_rate.cache_clear()             # manual invalidation

Know the limits before you use them for real data:

  • No TTL. An entry stays until it is evicted or you call cache_clear(). Fine for pure functions like fib; wrong for exchange rates.
  • Arguments must be hashable. A list or dict argument raises TypeError.
  • Shared mutable results. If the function returns a list and a caller changes it, every later caller sees the change.
  • Per process. With 8 web workers you have 8 separate caches, and invalidating one does not touch the others.
  • Not stampede-proof. The docs note the wrapped function may be "called more than once if another thread makes an additional call before the initial call has been completed and cached". Chapter 9 measures this.

Chapter 7 — Application caching patterns

When the cache is a separate system (Redis, Valkey, Memcached), there are four classic ways to wire it to the database:

Pattern Read path Write path Good for
Cache-aside (lazy loading) App checks cache; on miss, reads DB and fills cache App writes DB, then deletes the key The default choice; the cache can fail without losing data
Read-through App asks the cache; the cache library loads from DB on a miss Same as cache-aside Hides loading logic in one place
Write-through Always a hit after a write App (or cache) writes cache and DB together Data that is read right after it is written
Write-behind (write-back) Always a hit Write goes to the cache; DB is updated later in batches Very high write rates, if you can accept losing recent writes

TTLs

Every entry should have a TTL, even if you also invalidate on writes. The TTL is your safety net: if an invalidation is missed, the wrong value lives for at most one TTL. Choose it from the business question "how stale may this be?", not from performance alone. Add jitter (a small random extra) so that entries created together do not all expire together.

Invalidation strategies

  1. Expire only — rely on the TTL. Simplest; data is up to one TTL old.
  2. Delete on write — after the DB write succeeds, DEL the key. The next read reloads it. Prefer delete over "set the new value", because two concurrent writers can set values in the wrong order.
  3. Versioned keys — put a version in the key (user:42:v7) and bump the version on change. Old keys simply age out.
  4. Event-driven — the database emits change events and a worker deletes the matching keys. MongoDB Course 12 · Change Streams shows the event source.

Watch out: cache-aside has a known race. Reader A misses and loads the old row. Writer B updates the row and deletes the key. Reader A then writes the old value into the cache. It stays wrong until its TTL ends. Short TTLs keep the damage small; for critical data, add a version check or do not cache.

Chapter 8 — Redis and Valkey in practice

Which server?

  • Redis — Redis 7.2 and earlier are BSD-3-Clause. In March 2024 Redis moved to the source-available RSALv2 / SSPLv1 licenses. Since Redis 8 (May 2025), the open-source edition is tri-licensed: you may choose RSALv2, SSPLv1 or AGPLv3. AGPLv3 is the OSI-approved open-source option.
  • Valkey — a Linux Foundation fork announced in March 2024 that continued from Redis 7.2.4 under the BSD-3-Clause license. The 9.x line is current (9.0 in October 2025, 9.1 in May 2026).

For the commands in this post (GET, SET, DEL, EVAL), both behave the same, and the redis-py client works with either. Check the license and features against your own needs before you choose.

Start one locally with Docker. Bind the port to 127.0.0.1 only: an open Redis port without a password is a classic Security Misconfiguration.

The same two commands work in Git Bash and PowerShell:

1
2
3
docker run --rm -p 127.0.0.1:6379:6379 redis:8              # or: valkey/valkey:9
pip install redis

Cache-aside with jitter and a rebuild lock

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
# cache_aside.py — cache-aside with redis-py, jittered TTL and a rebuild lock
import json, random, secrets, sqlite3, time
import redis
 
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
db = sqlite3.connect("shop.db", check_same_thread=False)
db.execute("CREATE TABLE IF NOT EXISTS products "
           "(id INTEGER PRIMARY KEY, name TEXT, price_cents INTEGER)")
db.execute("INSERT OR IGNORE INTO products VALUES (1, 'Keyboard', 4999)")
db.commit()
 
TTL = 300                                    # 5 minutes
RELEASE = """
if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
end
return 0
"""
 
def load_from_db(pid: int) -> dict | None:
    time.sleep(0.2)                          # pretend the query is slow
    row = db.execute("SELECT id, name, price_cents FROM products WHERE id = ?",
                     (pid,)).fetchone()
    return None if row is None else dict(zip(("id", "name", "price_cents"), row))
 
def get_product(pid: int) -> dict | None:
    key = f"product:{pid}"
    for _ in range(40):                      # wait at most about 2 seconds
        cached = r.get(key)
        if cached is not None:
            return json.loads(cached)        # HIT ("null" means "known missing")
        token = secrets.token_hex(8)
        if r.set(f"lock:{key}", token, nx=True, ex=10):    # MISS: one rebuilder
            try:
                product = load_from_db(pid)
                ttl = 30 if product is None else TTL + random.randint(0, 60)
                r.set(key, json.dumps(product), ex=ttl)    # jitter spreads expiry
                return product
            finally:
                r.eval(RELEASE, 1, f"lock:{key}", token)   # delete only our own lock
        time.sleep(0.05)                     # someone else is rebuilding
    return load_from_db(pid)                 # give up waiting, go to the source
 
def update_price(pid: int, cents: int) -> None:
    db.execute("UPDATE products SET price_cents = ? WHERE id = ?", (cents, pid))
    db.commit()
    r.delete(f"product:{pid}")               # invalidate AFTER the write succeeds
 
if __name__ == "__main__":
    t = time.perf_counter(); print(get_product(1), f"{time.perf_counter() - t:.3f}s")  # miss ~0.2 s
    t = time.perf_counter(); print(get_product(1), f"{time.perf_counter() - t:.3f}s")  # hit ~0.000 s
    update_price(1, 4499)
    print(get_product(1))                    # price_cents 4499 after invalidation
    print(r.ttl("product:1"), "seconds left")  # between 300 and 360

What each piece does:

  • r.set(key, value, ex=ttl) stores the JSON with a TTL in seconds. ex=TTL + random.randint(0, 60) is the jitter.
  • Missing products are cached as null for 30 seconds (negative caching), so a flood of requests for a missing ID does not hit the database every time.
  • r.set(lock_key, token, nx=True, ex=10) is the lock: NX sets it only if nobody holds it, EX makes it expire if the holder crashes. The Lua script deletes the lock only if it still holds our token. This is the single-instance lock pattern from the Redis docs. It is fine for a cache, where two rebuilders once in a while only cost a little extra work. It is not a lock for protecting money.

Chapter 9 — Cache stampede and how to stop it

A cache stampede (or thundering herd, or dog-piling) happens when a popular key expires and hundreds of requests miss at the same moment. They all run the same slow query, the database slows down, requests pile up, and the cache never gets refilled. It often follows a deploy or restart (cold cache) or a batch of keys that were all created with the same TTL.

lru_cache shows the problem in miniature. With 20 threads asking for the same missing key at once, the slow function ran 20 times. The fixes:

Fix Idea
Jittered TTL ttl = base + random so keys created together expire at different times.
Locking Only the request that wins a lock recomputes; the others wait briefly and read the new value (Chapter 8).
Request coalescing (single-flight) In one process, concurrent callers for the same key share one in-flight computation.
Early refresh Recompute before expiry, so the key is never missing. Either a background job, or a random early refresh per request.
Serve stale Keep serving the old value while one worker refreshes (the idea behind stale-while-revalidate).

Single-flight TTL cache in pure Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# ttl_cache.py — a tiny TTL cache decorator with single-flight and jitter
import functools, random, threading, time
 
def ttl_cache(seconds: float, jitter: float = 0.1):
    def decorate(fn):
        store, locks, guard = {}, {}, threading.Lock()   # args -> (value, expires_at)
 
        @functools.wraps(fn)
        def wrapper(*args):
            hit = store.get(args)
            if hit and hit[1] > time.monotonic():
                return hit[0]                   # fresh hit, no locking
            with guard:
                lock = locks.setdefault(args, threading.Lock())
            with lock:                          # single-flight: one caller recomputes
                hit = store.get(args)
                if hit and hit[1] > time.monotonic():
                    return hit[0]               # another thread filled it meanwhile
                value = fn(*args)
                ttl = seconds * (1 + random.uniform(-jitter, jitter))
                store[args] = (value, time.monotonic() + ttl)
                return value
        return wrapper
    return decorate
 
calls = 0
 
@ttl_cache(seconds=2)
def load_profile(user_id: int) -> dict:
    global calls
    calls += 1
    time.sleep(0.3)                             # slow database query
    return {"id": user_id}
 
threads = [threading.Thread(target=load_profile, args=(7,)) for _ in range(50)]
for t in threads: t.start()
for t in threads: t.join()
print("50 concurrent callers, loads:", calls)   # 1

Fifty threads, one database call. The double check inside the lock is what makes it work: whoever waited finds the fresh value that the first caller stored. (The store never shrinks; a real version would also evict old keys.)

Probabilistic early refresh

Vattani, Chierichetti and Lowenstein ("Optimal Probabilistic Cache Stampede Prevention", VLDB 2015) propose that each reader flips a biased coin. The closer the key is to expiry, and the longer the recompute takes, the more likely the reader refreshes it early:

1
2
3
4
5
6
import math, random, time
 
def should_refresh_early(expires_at: float, delta: float, beta: float = 1.0) -> bool:
    # delta: seconds the last recompute took; beta > 1 refreshes earlier
    return time.time() - delta * beta * math.log(1.0 - random.random()) >= expires_at

With delta = 1 second, a quick simulation of 10,000 readers gives about 0% refreshes 10 s before expiry, about 5% at 3 s, about 37% at 1 s and about 82% at 0.2 s. Usually one reader refreshes the key shortly before it expires, and nobody ever sees a miss. Store delta next to the value so every reader can use it. (1.0 - random.random() avoids log(0).)

Practice Exercises

Exercise 1 — Choose the headers

Write a Cache-Control value for each: (a) logo.8c1f2e7a.svg, (b) the home page HTML, (c) GET /api/account for the logged-in user, (d) a public list of exchange rates updated every minute.

Solution

(a) public, max-age=31536000, immutable — the hash in the name changes with the content. (b) no-cache, plus an ETag, so every visit revalidates and usually gets a 304. (c) private, no-store (or private, no-cache if the data is not sensitive) — never in a shared cache. (d) public, max-age=60, stale-while-revalidate=30, or add s-maxage=60 if browsers should keep it shorter than the CDN.

Exercise 2 — no-cache or no-store?

A bank shows monthly statements at /statements/2026-08. The page is sent with Cache-Control: no-cache. Is that safe?

Solution

No. no-cache allows storing; it only forces revalidation before reuse. The statement can sit in the browser's disk cache on a shared computer. Use Cache-Control: no-store (with private as well for older caches).

Exercise 3 — Prove the 304

Using cache_server.py, show with curl that a second request for / can be answered with 304 and no body. Then change the body of / in PAGES, restart, and repeat with the old ETag. What happens and why?

Solution 
1
2
3
4
ETAG=$(curl -sI localhost:8000/ | grep -i '^etag' | cut -d' ' -f2 | tr -d '\r')
curl -s -o /dev/null -w "%{http_code} %{size_download}\n" -H "If-None-Match: $ETAG" localhost:8000/
# 304 0

After changing the body, the server computes a new ETag from the new bytes, so the old tag no longer matches and it returns 200 with the new body. That is exactly how a revalidating cache picks up changes.

Exercise 4 — Find the invalidation bug

1
2
3
4
5
def update_price(pid, cents):
    r.delete(f"product:{pid}")
    db.execute("UPDATE products SET price_cents = ? WHERE id = ?", (cents, pid))
    db.commit()
Solution

The key is deleted before the write. Between the delete and the commit, another request can miss, read the old row and put it back in the cache, where it stays for a full TTL. Also, if the update fails, the cache was cleared for nothing. Write first, commit, then delete. Keep a TTL as the safety net for the remaining race.

Exercise 5 — Measure a stampede

Decorate a function that sleeps 0.3 s with functools.lru_cache, call it from 20 threads at once with the same argument, and count the calls. Then swap in ttl_cache(seconds=60) from Chapter 9.

Solution 
1
2
3
4
5
6
7
8
9
10
11
12
13
import threading, time
from functools import lru_cache
 
n = 0
@lru_cache
def slow(x):
    global n; n += 1; time.sleep(0.3); return x
 
ts = [threading.Thread(target=slow, args=(1,)) for _ in range(20)]
for t in ts: t.start()
for t in ts: t.join()
print(n)   # 20: every thread missed before the first call finished

With @ttl_cache(seconds=60) instead of @lru_cache, n is 1: the other 19 threads wait on the per-key lock and then read the stored value.

Exercise 6 — Early-refresh odds

In should_refresh_early, the chance of refreshing is exp(-time_left / (delta * beta)). With delta = 1 and beta = 1, what is the chance 2 seconds before expiry? What does beta = 2 do?

Solution

exp(-2) ≈ 0.135, so about 13.5% of readers at that moment trigger a refresh. With beta = 2 the chance becomes exp(-1) ≈ 0.37: refreshes start earlier. Larger beta means fewer misses but more total recomputes.

Cheat-sheet

Need Use
Hashed static asset Cache-Control: public, max-age=31536000, immutable
HTML page Cache-Control: no-cache + ETag → cheap 304
Personal data private, no-cache, or no-store if sensitive
CDN longer than browser max-age=60, s-maxage=3600 or CDN-Cache-Control (RFC 9213)
Serve stale while refreshing stale-while-revalidate=N (RFC 5861)
Check headers / revalidate curl -I URL; curl -H 'If-None-Match: "etag"' URL → 304 (PowerShell: curl.exe)
Cache busting Content hash in the file name + manifest
Pure function memo @functools.cache / @lru_cache(maxsize=N)
Default app pattern Cache-aside: read → miss → DB → SET key value EX ttl
On write Write DB, commit, then DEL key
Spread expiry TTL + random jitter
One rebuilder SET lock:key token NX EX 10 + compare-and-delete release
In-process coalescing Per-key lock + double check
Refresh before expiry XFetch: now - delta*beta*log(rand) >= expiry
Redis license (8.x) RSALv2, SSPLv1 or AGPLv3; Valkey is BSD-3-Clause

What's next

Dev Essentials 14 · API Reliability covers what happens when the thing behind your cache is slow or down: timeouts, retries with backoff, idempotency keys, rate limits and queues.

References

Note: this post is in progress. Planned additions: an nginx reverse-proxy cache demo, cache metrics (hit ratio, evictions) with Redis INFO, and a Next.js ISR walk-through.