Software Architecture / Evolution: capacity, migration, validation, and review / Lesson 14

Make caching a freshness and failure decision

Design cache keys, invalidation, stampede protection, and authoritative fallback for workshop queries.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

State what may be stale and which operations must remain authoritative.
Explain cache-aside races and safe key boundaries.

Cache a defined query rather than an undefined object

A cache stores a reusable result to avoid repeated work, but it also introduces another state representation. Define the query, freshness tolerance, sensitivity, and miss behavior before selecting a product. A public workshop description can tolerate a different delay from a permission decision or remaining-seat allocation. A cache hit should preserve the same access policy as the authoritative path. Caching is valuable when repeated work and acceptable reuse are demonstrated; a low hit rate or large invalidation burden may make the extra state counterproductive.

Include every result-changing dimension in the key

If a workshop description differs by language, a key containing only workshop identity can serve Arabic text to an English request or vice versa. Tenant, permission scope, representation version, and query parameters can also matter. Avoid placing secrets or personal data directly in observable key names. The example includes a representation version and language for a public query. It does not make private data public or define authorization policy. Review normalization carefully so equivalent inputs share a key and different authorized results never collide.

python A bounded public-query key model

def workshop_key(workshop_id, language):
    if language not in {"en", "ar"}:
        raise ValueError("unsupported language")
    return f"workshop-description:v2:{workshop_id}:{language}"

print(workshop_key("ws-42", "en"))
print(workshop_key("ws-42", "ar"))

Expected output

workshop-description:v2:ws-42:en
workshop-description:v2:ws-42:ar

The two languages produce distinct keys. This complete program uses fixed public identifiers only and contacts no cache service. Other result dimensions must be added when the real contract requires them.

Invalidation is a concurrency protocol

Cache-aside reads the cache, loads the authoritative store on a miss, and writes the result back. A concurrent update and invalidation can race with a slow reader that later repopulates an old value. A time-to-live bounds one kind of staleness under specified assumptions but does not guarantee immediate correctness. Versioned values, conditional writes, event-driven invalidation, or bypassing the cache for sensitive operations may address different scenarios. Explain the exact ordering guarantee needed rather than claiming that deleting a key universally solves cache consistency.

Bound miss work and choose failure behavior

When a popular key expires, many callers may rebuild it simultaneously and overload the authoritative store. Request coalescing, bounded refresh concurrency, and carefully chosen expiry variation can reduce this stampede, with implementation-specific costs. Decide whether stale data may be served during refresh or whether the request must fail honestly. A cache outage should not automatically send unlimited parallel requests to a vulnerable database. The design note names separate policies for descriptions and seat allocation, illustrating that one global fallback rule is usually too vague.

yaml Different cache rules for different promises

workshop_description:
  freshness: bounded stale display permitted by product policy
  miss: coalesced bounded refresh
  cache_failure: bounded authoritative read or explicit failure
seat_allocation:
  cache_authority: never
  decision: authoritative protected write path

The policy intentionally contains no universal expiry duration. Product freshness requirements and measured source capacity determine concrete values in an implementation.

Test cold, stale, and failed-cache paths

A useful isolated test suite covers a cold miss, a hit, an update racing a refresh, simultaneous expiry, cache unavailability, and authorization-sensitive variants. Inspect both returned content and source request volume. A high hit ratio alone does not prove users receive correct or current information. Make cache ownership and invalidation events visible in documentation, and include deletion and retention handling for sensitive data. This lesson executes only key construction; it does not start a cache, measure hit ratios, or verify any vendor consistency guarantee.

Common errors and fixes

Arabic and English descriptions share one cached result.

Why it happens: Language was absent from a result-changing key.
How to fix it: Add language and migrate or invalidate the old representation safely.
Prevent it next time: List all result dimensions before designing keys.

A stale seat count authorizes the last reservation.

Why it happens: Display optimization crossed the authoritative write boundary.
How to fix it: Validate allocation atomically at the reservation owner.
Prevent it next time: Classify every cached value by permitted use.

Cache failure overwhelms the database.

Why it happens: Fallback requests were unbounded and uncoordinated.
How to fix it: Bound refresh and authoritative fallback concurrency.
Prevent it next time: Test a failed cache and an expired popular key in the lab.

Check your understanding

1. Does a sixty-second expiry guarantee every response is at most sixty seconds stale?

Show worked solution

Not without precise assumptions about when the value was read, written, refreshed, and served. A delayed reader can repopulate old data, and stale-serving policies can extend reuse. Define the freshness clock and update protocol, then test the relevant race. Do not infer a complete guarantee from an expiry setting alone.

2. Why can a lower hit ratio still be an acceptable design?