Caching

Atom is designed to sit on the auth path of every downstream service. Without a cache, each request costs three or four Postgres round trips, one of which is a recursive CTE. This page explains the Redis cache that sits in front of those reads: what it stores, when it invalidates, and why it can be trusted with security-sensitive decisions.

Goal

For every request arriving at Atom:

• JWT auth needs a three-way join across sessions, entities, and tenants.
• API-key auth needs the same shape across credentials, entities, and tenants, plus a KDF verification.
• Every authorization decision needs subject_effective_grants(uuid) — a recursive CTE that flattens direct policies, role assignments, and group membership.

The cache targets these hot reads. Postgres remains the single source of truth; the cache is optional at every call site.

Design Principles

1. Cache inputs, never decisions. The permit/deny outcome is never cached, so a policy change takes effect on the next request without touching a combinatorial (subject, action, object) space.
2. Correctness over hit rate. A revoked token, disabled entity, or removed grant must stop working immediately. Every mutation that could affect a cached value invalidates the affected keys through a race-safe barrier.
3. Fail-safe reads. If Redis is unreachable, reads fall through to Postgres. Auth still works — just slower.
4. Fail-refused writes. If Redis is unreachable during a security-sensitive Postgres mutation, the mutation is refused. Committing without being able to invalidate would risk serving stale grants after a revoke.
5. Optional at every layer. Every call site tolerates cache: None. Removing Redis is a config change, not a code change.

cache: None means caching is not configured, and it is the one state in which every mutation guard degrades to a pass-through. An enabled cache that merely cannot reach Redis is a different state and never collapses into None — otherwise a replica that booted during an outage would mutate grants, sessions, and credentials without invalidating entries its peers were still serving, and a revoke on one replica would stay authorized on another. Unreachable Redis is a runtime condition: the client is retained, reads degrade to misses, and begin fails so security-sensitive mutations are refused until Redis returns.

What Is Cached

Six categories, each keyed by a UUID under the atom:v1: namespace.

Payloads are positional MessagePack. Exact v1 bytes are frozen in api/v1/cache-wire-v1.json; changing a field order, enum spelling, nested shape, or encoding requires a new logical key version and rollout plan.

What is not cached

• Passwords — never used on the request path. Used only during login, which mints a JWT; the JWT then uses the Session cache.
• Plaintext API-key secrets — only the hash used to verify them.
• The authorization decision itself — the PDP evaluates conditions per request against fresh (or freshly cached) grants.
• Audit writes — always go to Postgres.
• Tombstones. The entity and tenant entries carry no deleted_at. Both miss loaders already filter deleted_at IS NULL, so a cached copy would be empty by construction and any check against it a no-op that merely looked like a tombstone check. Denying a subsequently soft-deleted entity or tenant is the delete path's invalidation duty.

Request Flow

The auth hot path batches the keys it can into one atomic Lua round trip on a single pooled connection. Issued one at a time, three lookups would mean three pool acquisitions and three serial round trips, each bounded by the operation timeout, before any request work started.

Only keys with no data dependency on each other can share a round trip. For JWT auth, session, entity, and tenant are all known up front — the tenant key comes from the token's tid claim — so all three go together. For API-key auth the credential key is read first and alone because it gates everything after it, and the tenant key is derived from the entity's current tenant rather than from the credential, so it cannot be batched with it.

Consistency Model

Every cached entry is a Redis hash with these fields:

• i — the random namespace incarnation that owns the hash.
• v — an integer version, bumped on every mutation that could affect the entry.
• dirty — the count of live mutation leases.
• p — the serialized payload, present only when the entry holds a valid value.
• lease:<uuid> — one exact persistent token per in-flight mutation.

Two persistent namespace keys hold a random incarnation marker and a global mutation_epoch. Every process remembers the incarnation it observed during startup, and every Lua script atomically verifies it. If Redis loses or replaces that marker, the process permanently stops cache reads, refuses protected mutations, and fails readiness until restart. Initialization succeeds only when both the marker and epoch are absent; a missing marker with a surviving epoch is treated as a damaged or globally fenced generation.

Four atomic Lua scripts implement a per-key mutation barrier.

Read path

Write path

The races this prevents

1. Read-before-mutation. A reader observes version N, starts loading from Postgres; a mutation runs to completion, bumping to N+2. The reader's try_populate presents N — rejected on version mismatch.
2. Read-during-dirty-window. A reader lands while dirty=1, observes the post-begin version. end's second version bump ensures that observed version is stale by the time try_populate runs — rejected either by the dirty check (if still dirty) or the version check (if end already ran).
3. Lost-invalidation. If end never runs (crash), the exact lease and dirty hash remain persistent, forcing Postgres fallback until a full-drain operator repair.
4. Cross-key poisoning during populate. A miss loader whose returned payload describes a different key than the one being populated must never write across keys — populates only apply when the observed version's key matches the key the payload describes.
5. Cleanup destroying a live barrier. Discarding a corrupt payload must not take v and dirty with it. Deleting the whole hash would erase a concurrent mutation's barrier: the next reader would find an absent key, observe version 0, load pre-commit state, and populate it successfully. discard is version-guarded and clears only p; end also clears p defensively.
6. Redis state loss. A reader that started before a flush/replacement presents its remembered incarnation when its Postgres load finishes. The fresh namespace has a different marker, so populate is rejected and the old process permanently latches unsafe.
7. Unsafe topology or a lost exact lease. Atom globally poisons the namespace marker, making every peer fail closed. If the poison write fails, it deletes the marker and the surviving epoch prevents in-place reinitialization.

Invalidation Map

Tenant creation is on this list because create_tenant bootstraps a tenant-admin role, role assignment, and membership for the creator in the same transaction — it grows the creator's own grant set, and the capability gate immediately above it has just warmed that exact key. purgeTenant performs no invalidation: it is reachable only for an already-soft-deleted tenant, whose soft delete invalidated TenantStatus and the members' sessions, so the tenant is already denied at the lifecycle check that runs before grant matching.

Race-safe enumeration

Group and role invalidations must enumerate who is affected — every entity in a group closure, every assignee of a role. If enumeration happens outside a transaction, a concurrent add_group_member can slip a new member in between enumeration and mutation. That new member's grants key is never invalidated, and a stale entry survives until TTL.

The enumeration functions therefore lock the relevant rows FOR UPDATE inside the caller's transaction, in id-sorted order so they cannot deadlock against themselves. The guarded_tx_mutation helper wires that ordering — open transaction, lock and enumerate, begin barrier, mutate, commit, end barrier — in one place so every call site does it identically.

Failure Modes

Configuration

All knobs live under the ATOM_CACHE_* prefix:

• ATOM_CACHE_MODE — disabled, prepare (barriers only), or enabled.
• ATOM_CACHE_ENABLED — deprecated alias used only when mode is absent.
• ATOM_CACHE_REDIS_URL — connection URL.
• ATOM_CACHE_NAMESPACE — required deployment/database-unique prefix.
• ATOM_CACHE_INITIALIZE_NAMESPACE — one-startup initializer for a new/empty namespace. Use only after the entire Atom fleet and all in-flight requests are drained, then unset it.
• ATOM_CACHE_POOL_MAX_SIZE — max Redis connections.
• ATOM_CACHE_FAIL_FAST_ON_STARTUP — true aborts startup when Redis is unreachable. Default false.
• ATOM_CACHE_CONNECT_TIMEOUT_MS — startup PING timeout.
• ATOM_CACHE_OP_TIMEOUT_MS — per-operation timeout.
• ATOM_CACHE_TTL_<CATEGORY>_SECS — one per category (session, entity_status, tenant_status, credential, credential_ceiling, grants). Each must be greater than zero and no more than 86400 seconds (24h).

Clean payload hashes expire by category TTL. Dirty barriers, the incarnation, and the epoch do not expire, but the supported Redis itself is intentionally ephemeral: one dedicated standalone primary, maxmemory-policy=noeviction, save "", appendonly no, no replicas, no Cluster, and no automatic failover. Atom verifies this with CONFIG GET and INFO replication. Its ACL needs those read commands, but should not grant CONFIG SET, flush/migration/restore, or replication reconfiguration.

Every Redis restart, flush, replacement, or barrier repair requires: stop traffic; fully terminate/drain every Atom process and in-flight request; empty the namespace; start one process with ATOM_CACHE_INITIALIZE_NAMESPACE=true; wait for readiness; unset it; then start the remaining fleet. A rolling restart is not sufficient. Direct SQL, backfills, or migrations that change cached inputs must use the same barrier protocol or run under this full-drain/fresh-namespace procedure.

Metrics

Category labels are fixed enum variants, so cardinality is bounded.

The populate metric splits applied (write took) from stale (rejected by the barrier: dirty entry, or the version moved since the caller's lookup). A hit rate stuck at zero can then be told apart from every write being rejected by a stuck barrier — otherwise indistinguishable.

Extending

The cache client (Redis pool, Lua scripts, barrier, timeouts, metrics) is fully generic over any serde type. Only the registry of categories is centralised in the codebase — one enum, one TTL config struct, one file of key builders — so that:

• Metrics label cardinality stays bounded.
• All TTLs are auditable in one file.
• No handler can silently invent a new cache category that collides with an existing one.

Adding a new cached category is a five-step, one-file-at-a-time change: extend the category enum, add a TTL field, add a key builder, wrap read sites in the cache-aside helper, and wrap write sites in the invalidation helpers.