Agent LLM Call Persistence

Status: accepted (2026-07-08) · implemented (Phases 1–5).

What was decided

Capture every real-inference hosted-agent turn's actually-rendered prompt + raw LLM response, so a developer can reconstruct "what was this agent told, decided, and why" for any recent hand — the data substrate of the "agent lab" family (agent decision inspection, training sandbox, prompt-safety review).

Four decisions carried the design:

Store the rendered bytes (deduped), not reconstruct-from-source. The read must be byte-identical to what the model saw — and re-rendering at read time can't stay identical across assembler/skill changes. So we store the actual strings, split into a content-addressed repeated envelope (base prompt + strategy + skill sections, hashed and stored once per distinct value) + a per-turn remainder (game state + hand history). Reconstruction concatenates them exactly. This gave ~190× dedup live (1,353 turns → 7 envelope blobs) at zero fidelity cost, and it survives skill/prompt evolution for free — a change mints a new envelope hash; old captures reconstruct against their original.
A dedicated Redis process for the 24h hot buffer, isolated from the turnstore and engine Redis. Redis maxmemory/eviction is per-server, so co-locating a high-churn evictable buffer with must-keep turnstore state risks evicting the turnstore under pressure. The capture Redis runs volatile-ttl; the turnstore runs noeviction. That policy difference is the reason they're separate.
Best-effort capture that never blocks a turn — fire-and-forget + bounded retry + error-log at the runInference core, on the success path only (the check-else-fold fallback is excluded by construction).
promote(hand) → durable Postgres, pin-only. The buffer is disposable (24h TTL); anything worth keeping is pinned to agent_call_records. Durable growth tracks pinned hands, not turns.

A bare admin viewer (/agent-capture) renders it: a collapsible recent-hands history (from agent_inference_charges) → hand-ID lookup → per-turn prompt + response + action + fallback badge, with "Keep this hand" (promote).

Tradeoffs accepted

Storage vs. fidelity: we store rendered bytes (more storage) to guarantee byte-identity, rather than the cheaper reconstruct-from-OHH that would drift.
One more Redis process to provision (local compose + a dedicated Fly app on staging/prod) in exchange for eviction isolation from the turnstore.
A boot-required env var (AGENT_CAPTURE_REDIS_URL) — a misconfigured deploy fails loud rather than capturing silently to nowhere; the cost is a pre-merge provisioning gate (dedicated Redis + Doppler var per environment).
Buffer/index decoupling: the recent-hands list (persistent) outlives the capture buffer (24h), so it can list hands whose turns are already gone — the viewer signals this (fallback-only markers, honest "nothing to pin" on expiry), surfaced by the falsification pass.

Agent LLM Call Persistence ​

What was decided ​

Tradeoffs accepted ​

See also ​

Agent LLM Call Persistence

What was decided

Tradeoffs accepted

See also