What is per-module cache ownership in a modular monolith?
Per-module cache ownership is a simple rule: each module in the monolith owns one Redis key namespace, and only that module's code may write to or invalidate keys under it. The billing module owns everything under billing:. The catalog module owns catalog:. If catalog needs billing's cached data to change, it asks billing to invalidate it. It never deletes billing's keys directly.
Keys follow a fixed shape, module:entity:version:id. A cached product is catalog:product:v7:12345, and a cached invoice is billing:invoice:v3:98012. The module segment names the owner, and the entity segment names the data type. The version segment lets the owner retire a whole set of keys at once, which a later section covers. The id identifies the record.
Redis has no real namespaces inside a logical database. A colon in a key name means nothing to the server. Any client connected to that database can read, overwrite or delete billing:invoice:v3:98012, whatever prefix it carries. The prefix is a convention that helps humans read keys and lets tooling group, count and attribute them. It does not isolate anything. Enforcement has to come from your code structure and review process.
Teams adopt the rule because it narrows where invalidation can happen. Without it, deletes end up scattered across handlers, jobs and other modules, and nobody can say which writes clear which entries. With it, you can audit a module's invalidation logic by reading that one module. The pattern suits teams running a modular monolith with real module boundaries and one shared Redis instance, where several modules share the same keyspace.
How does cache-aside work inside a module, and where does it race?
Within a module, the read path is simple: look up the key, and on a miss load from the database and write the result back with a TTL. The write path is where teams go wrong. Commit to the source of truth first, then delete the cache entry. Do not set it.
Here is why setting fails. Writer A commits version 1 and writer B commits version 2. B sets the cache to 2, then A, delayed by a GC pause, sets it to 1. The cache now holds a stale value with a full TTL ahead of it. A delete carries no value, so the order of concurrent deletes doesn't matter. The next reader loads whatever the database holds at that moment.
Delete-after-write still leaves one race:
- Reader R misses and reads version 1 from the database.
- R stalls before writing to the cache.
- Writer W commits version 2 and deletes the key.
- R resumes and writes version 1 into the now-empty slot.
The stale value survives until something evicts it.
| Mitigation | What it guarantees | Cost |
|---|---|---|
| Short TTL backstop | Staleness bounded by the TTL | More misses and database load |
| Versioned keys | Readers ask for the current version, so old keys are never read | Readers need a cheap way to learn the current version |
| Conditional write (compare version or timestamp atomically, e.g. in a Lua script) | A cache write never moves an entry backwards | Every cached value must carry a version from the source of truth |
None of these gives you strong consistency for free. You either carry versions through every write, or you accept staleness bounded by the TTL and size the TTL to match what the module can tolerate.
import json
import redis
r = redis.Redis(host="cache.internal", port=6379, decode_responses=True)
TTL = 300 # backstop: bounds staleness if anything below is bypassed
# Atomic conditional write: refuse to move an entry to an older version.
# Empty data means "tombstone": remember the version, drop the value.
CAS = r.register_script("""
local cur = redis.call('HGET', KEYS[1], 'v')
if cur and tonumber(cur) > tonumber(ARGV[1]) then return 0 end
if ARGV[2] == '' then
redis.call('HDEL', KEYS[1], 'data')
redis.call('HSET', KEYS[1], 'v', ARGV[1])
else
redis.call('HSET', KEYS[1], 'v', ARGV[1], 'data', ARGV[2])
end
redis.call('EXPIRE', KEYS[1], ARGV[3])
return 1
""")
def key(order_id):
return f"orders:order:{order_id}" # module-owned namespace
def get_order(db, order_id):
cached = r.hget(key(order_id), "data")
if cached is not None:
return json.loads(cached)
row = db.fetch_order(order_id) # row carries 'version' from the DB
# A slow reader holding an old version is rejected here.
CAS(keys=[key(order_id)], args=[row["version"], json.dumps(row), TTL])
return row
def update_order(db, order_id, changes):
new_version = db.update_order(order_id, changes) # commit first
# Invalidate with a versioned tombstone, not a plain DEL: an empty slot
# would let a stalled reader write version N-1 back in.
CAS(keys=[key(order_id)], args=[new_version, "", TTL])
Cache-aside in a module-namespaced key. Writers commit to the database first and then invalidate with a versioned tombstone. A Lua compare-and-set rejects any write that carries an older version, which closes the slow-reader race that a plain delete leaves open.
How do version counters invalidate a whole namespace without KEYS or SCAN?
Pattern deletes such as "remove everything under billing:*" are the usual first attempt. A generation counter replaces them. Each module owns one integer key, for example billing:ns:ver. Every data key embeds that integer: billing:v42:invoice:1881. The read path is two lookups. First GET the version, then build the data key from it and GET that. On a miss, load from the source of truth and SET the result with a TTL. To invalidate the whole namespace, run one INCR on the version key. Readers immediately build v43 keys, and every v42 entry becomes unreachable. The two lookups are sequential because the second depends on the first. Some teams cache the version in process for a second or two to save the round trip, which accepts that much extra staleness.
The alternative is dangerous because Redis executes commands on a single thread. KEYS walks the entire keyspace in one call and blocks every client while it runs, including other modules sharing the instance. SCAN avoids the long block by iterating in small batches with a cursor, but it is not a point-in-time snapshot. Keys created during the sweep may be missed, and some keys can be returned more than once. Concurrent writers can also repopulate stale entries behind the cursor. On a large keyspace a full sweep still touches every key, across many round trips.
The cost of the counter approach is memory. Orphaned v42 keys stay resident until their TTL expires, so every data key needs a TTL. Without TTLs, each bump leaks a full generation of entries. Keep the version key itself free of a TTL and use a volatile-* eviction policy. That way eviction never removes the version key. If it were removed, readers could fall back to an old generation number and resurrect stale data.
import json
import time
import redis
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
DATA_TTL_SECONDS = 600 # every data key expires, so orphaned generations are reclaimed
def ns_version(module: str) -> int:
ver_key = f"{module}:ns:ver" # no TTL on this key, ever
ver = r.get(ver_key)
if ver is None:
# Seed from wall-clock millis instead of 0 so a lost version key
# cannot fall back to an old generation and resurrect stale data.
r.set(ver_key, int(time.time() * 1000), nx=True)
ver = r.get(ver_key)
return int(ver)
def data_key(module: str, version: int, entity: str, entity_id: str) -> str:
# e.g. billing:v42:invoice:1881
return f"{module}:v{version}:{entity}:{entity_id}"
def get_invoice(invoice_id: str, load_from_db) -> dict:
# Lookup 1: current generation. Lookup 2 depends on it, so they are sequential.
key = data_key("billing", ns_version("billing"), "invoice", invoice_id)
cached = r.get(key)
if cached is not None:
return json.loads(cached)
value = load_from_db(invoice_id) # miss: go to the source of truth
r.set(key, json.dumps(value), ex=DATA_TTL_SECONDS)
return value
def invalidate_module(module: str) -> int:
# One O(1) command replaces KEYS/SCAN + DEL. Old-generation keys become
# unreachable and linger only until their TTL expires.
return r.incr(f"{module}:ns:ver")
# Server config to pair with this: maxmemory-policy volatile-lru
# (evicts only keys with a TTL, so the version key is never evicted)
A redis-py read path that builds generation-stamped keys from a per-module version counter, caches misses with a TTL, and invalidates a whole module's namespace with a single INCR.
How should cross-module invalidation and transaction boundaries work?
Suppose the orders module caches order summaries that embed the customer's display name, which the customers module owns. When a customer is renamed, the customers module publishes a CustomerRenamed event. The orders module handles it by bumping its own namespace version or deleting the specific summary keys. The customers module never deletes keys under the orders prefix. If it did, it would need to know the orders module's key layout, and every change to that layout would break invalidation without any error.
Publish only after commit
Invalidation must follow a successful commit. If the event fires inside the transaction, a concurrent reader can miss the cache, read the old row (the update is not yet visible to it), and write that old value back. The commit then lands, and nothing triggers another invalidation, so the stale entry stays until its TTL expires.
An after-commit hook from your ORM or transaction manager is the simplest option. It runs in-process with almost no delay. However, if the process crashes between commit and dispatch, the event is lost.
A transactional outbox writes the event row in the same transaction as the data change, and a relay publishes it later. This makes delivery durable and at-least-once. Invalidation is idempotent, so duplicate deliveries cause no harm. The costs are a relay to operate and a window of eventual consistency.
Strict ownership also means two modules may cache overlapping views of the same entity, which duplicates memory and invalidation work. That cost buys independent schema changes. If the overlap keeps growing, the owning module should expose the shared read model through its own query API.
import redis
from sqlalchemy import event, update
from sqlalchemy.orm import Session
r = redis.Redis(host="localhost", port=6379)
_handlers = {} # minimal in-process event bus
def subscribe(name, handler):
_handlers.setdefault(name, []).append(handler)
# --- customers module: owns the row, knows nothing about orders:* keys ---
def rename_customer(session: Session, customer_id: int, new_name: str):
session.execute(update(Customer).where(Customer.id == customer_id)
.values(display_name=new_name)) # Customer: your mapped model
# Queue the event; never dispatch inside the open transaction.
session.info.setdefault("pending_events", []).append(
("CustomerRenamed", {"customer_id": customer_id}))
@event.listens_for(Session, "after_commit")
def dispatch_after_commit(session):
# Runs only after the commit succeeds, so readers now see the new row.
# In-process: if the process dies right here, the event is lost.
# Use an outbox row written in the same transaction when that matters.
for name, payload in session.info.pop("pending_events", []):
for handler in _handlers.get(name, []):
handler(payload)
@event.listens_for(Session, "after_rollback")
def drop_on_rollback(session):
session.info.pop("pending_events", None) # nothing committed, nothing to invalidate
# --- orders module: invalidates only keys under its own prefix ---
def on_customer_renamed(payload):
# Summary keys embed this per-customer version, e.g.
# orders:summary:{order_id}:v{ver}. Bumping it orphans old entries (TTL cleans up).
# INCR is safe to repeat, so at-least-once delivery from an outbox is fine.
r.incr(f"orders:customer_ver:{payload['customer_id']}")
subscribe("CustomerRenamed", on_customer_renamed)
The customers module queues a CustomerRenamed event that a SQLAlchemy after_commit hook dispatches only once the commit succeeds, and the orders module reacts by bumping a version counter under its own Redis prefix, never touching keys it does not own.
How do you enforce cache boundaries and make ownership visible in operations?
The wrapper is the enforcement point. Each module gets a cache client constructed with its namespace, for example billing:, and every call (get, set, delete, version bump) checks that the key starts with that prefix before touching Redis. A key outside the prefix raises an error instead of quietly succeeding. Failing loudly matters, because a silent cross-module write is the exact bug this structure exists to prevent.
Convention alone does not hold. Under deadline pressure, someone imports the raw Redis client because the wrapper lacks a command they need, and nothing stops them. Back the wrapper with a lint rule that forbids importing the underlying client outside the cache package. Add a review check that flags any new code path constructing a client for another module's namespace. When the wrapper is missing a command, add it to the wrapper. Do not bypass it.
The wrapper also gives you per-namespace telemetry. Tag hits, misses and bytes written with the module name. Redis reports memory and evictions only globally, so you need two extra steps to attribute them. For memory, periodically sample key sizes per prefix. For evictions, subscribe to keyspace eviction notifications and count them by prefix.
This saves time during incidents. A staleness report starts with one owner and one invalidation path, so nobody has to search the whole codebase for writers. When eviction rates climb, on-call can see which module's footprint grew and page that team directly.
import redis
from prometheus_client import Counter
# Every metric carries the owning module, so dashboards group by owner.
CACHE_OPS = Counter("cache_ops_total", "Cache ops by module", ["module", "op", "result"])
CACHE_BYTES = Counter("cache_bytes_written_total", "Bytes written by module", ["module"])
class CrossNamespaceError(ValueError):
"""Raised when a module touches a key outside its own prefix."""
class ModuleCache:
def __init__(self, client: redis.Redis, module: str):
self._r = client # the only reference to the raw client; lint forbids importing it elsewhere
self.module = module
self.prefix = f"{module}:"
def _check(self, key: str) -> str:
# Fail loudly: a silent cross-module write is the bug this wrapper exists to prevent.
if not key.startswith(self.prefix):
raise CrossNamespaceError(f"module {self.module!r} may not touch key {key!r}")
return key
def get(self, key: str):
value = self._r.get(self._check(key))
CACHE_OPS.labels(self.module, "get", "hit" if value is not None else "miss").inc()
return value
def set(self, key: str, value: bytes, ttl_seconds: int):
self._r.set(self._check(key), value, ex=ttl_seconds)
CACHE_BYTES.labels(self.module).inc(len(value))
def delete(self, key: str):
return self._r.delete(self._check(key))
def bump_version(self, key: str) -> int:
# Version counters live in the owner's namespace too, e.g. billing:invoice:ver
return self._r.incr(self._check(key))
billing_cache = ModuleCache(redis.Redis(host="cache.internal", port=6379), "billing")
billing_cache.get("billing:invoice:42") # allowed
# billing_cache.get("orders:cart:7") # raises CrossNamespaceError
A per-module Redis client wrapper that rejects keys outside the module's prefix and tags hit, miss and bytes-written metrics with the owning module's name.
TTL policy, jitter and stampede protection per module
TTL is a staleness budget, and each module sets its own. A pricing module whose writers always invalidate on commit can use long TTLs, because expiry is only a backstop for missed invalidations. A recommendations module that never invalidates relies on TTL alone, so its TTL is the maximum staleness users will see. Keep these values in the module's own configuration so a reviewer can see each budget next to the code that depends on it.
Fixed TTLs cause synchronized expiry. Keys written together, after a deploy, a cache flush or a warm-up job, expire together, and the database takes the combined miss load. Add random jitter to every TTL.
Illustrative estimate (assumed inputs): a warm-up job writes 10,000 keys within 60 seconds, each with a 3,600-second TTL. Without jitter, all 10,000 expire in the same 60-second window, about 167 misses per second. With ±10% jitter, expiry spreads across roughly 780 seconds, about 13 misses per second.
Jitter does not protect a single hot key. When that key expires, every concurrent request misses at once. There are two common fixes:
- Request coalescing. Concurrent requests in one process wait for a single loader. This is simple and needs no lock. It deduplicates only within that process, so N instances can still send N loads unless you coordinate across processes.
- Short-lived lock (
SETwithNXand an expiry). One caller rebuilds the value, and the others serve stale data or wait briefly. If the lock TTL is shorter than the rebuild time, two loaders run. If it is far longer and the holder crashes, waiters stall until the lock expires. Set the lock TTL from measured rebuild latency, and use locks only on keys hot enough to need them.
When should you avoid shared Redis, and what are the alternatives?
Prefixes organize keys; they do not isolate resources. All modules on one instance share memory, CPU, connections and failure. Under a maxmemory eviction policy, Redis picks victims across the whole keyspace, so one module that starts caching large serialized reports can push another module's hot session keys out. A slow command from one module also blocks everyone, because Redis runs commands on a single thread. Move off the shared instance when one module's memory growth, latency or availability needs differ from the rest.
| Option | Isolation | Portability | Typical fit |
|---|---|---|---|
| Shared instance, prefixes | None beyond naming | High | Early stage, similar workloads |
| Separate logical databases | Keyspace only; memory and CPU still shared | Low; unsupported in cluster mode, discouraged elsewhere | Rarely worth it |
| Separate instances | Memory, eviction, failure | High | Hot or large modules, differing SLAs |
| In-process cache | Full per process; no shared state | High | Small, read-mostly reference data |
Logical databases offer little isolation and lock you out of cluster deployments, so prefixes on shared instances, plus separate instances when a module needs them, is the more portable path.
The wrappers, invalidation events and naming conventions cost real effort up front. Compare that with auditing a codebase full of ad hoc DEL calls that reach across modules. Because every key a module touches goes through its own client and carries its own prefix and version counter, moving a module to a dedicated instance changes only that module's connection configuration. No call sites move, and no keys from other modules need to migrate.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to monolith cache invalidation, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.