skills·poker Docs

Appearance

2026-05-19 — Bulletproof Hosted Agents Phase 10 cash dogfood ​

Accepted 2026-05-10. Phase 10 closed 2026-05-19 with a documented bug carry-forward.

What happened ​

scripts/dev/run-cash-hosted-agents.ts drives 9 distinct hosted agents through the production hosted-agent-service code path, staggered 2s apart, each owned by a different harness wallet slot. The point of the dogfood was to validate the multi-owner cash seat flow (permit → /api/deposit → /seat) at concurrency before Phase 11's SnG repeat.

First run failed completely. Original script ordering followed useSeatAgent's documented order: /api/deposit then /seat. Every agent's credit-chips returned 404 no_seat because the deposit's await blocked on chain confirmation; by the time credit-chips fired, the agent's WS sit_down had not yet placed a reserve. 9 reserves placed AFTER their deposits, all timed out at 30s. No agents seated.

Second run flipped the order: /seat first (then 250ms beat, then /api/deposit). 7 of 9 seated cleanly. The remaining 2 hit table_frozen freezeReason=divergence: the custody invariant correctly detected that 7000 chips were on-chain but only 6000 were in PM seat totals — one of the 7 seats had a chip-account ghost.

Hands played to completion for the 7 seated agents. The hosted-agent-service's /seat flow, JWT auth, agent-token verification, and admin-server agent-id routing all worked. The integration's load-bearing parts validated.

Findings ​

Bug 15 — deposit→credit-chips race surfaces at concurrent scale ​

The custody invariant ALWAYS-ON safety check did its job. pool=7000, seatTotal=6000, delta=1000 is reproducible and traceable to a timing race between admin-server's credit-chips call and game-server's WS sit_down placing the reserve. Filed as a separate follow-up.

This is exactly the class of bug Phase 10 dogfood was supposed to surface. Structured tests don't drive 9 concurrent seat flows through the full production WS + on-chain path; only an actual dogfood at scale does. Phase 10 succeeded by finding this.

useSeatAgent's documented order races ​

Both the Hosted Agents guide and the production hook order the calls /api/deposit THEN /seat. That works in the UI ONLY because the browser's network stack interleaves the calls (deposit's chain-confirmation latency leaves a window where /seat lands first on game-server). Script-side that ordering never works — await blocks make the race deterministic in the wrong direction.

Two repair paths in the Bug 15 brief: (a) admin-server's deposit handler short-polls for the reserve before falling through to the 30s reconciler tick, (b) flip the documented order to /seat first. Both are tractable; (a) is more conservative.

Faucet amount caps the buy-in ​

ensureHarnessAccountFunded calls faucetTo once, which mints 1000 CTN. A buy-in larger than 1000 silently fails the deposit with a registerAndDeposit revert. First attempt used maxBuyIn=1000 and --buy-in=1500 → all 9 reverted with "Internal error" before I traced it to balance. Fix: either loop the faucet OR document the cap.

Once-frozen tables have no operator unfreeze path ​

After the custody invariant fires, the table is frozen in DB. Subsequent deposits 409. There's no admin-server endpoint to "reconcile + clear" today. For dev that's annoying; for prod that's a real operations gap. Captured in the Bug 15 brief as a sequenced repair.

What this closes ​

Closed: Phase 10 of bulletproof-hosted-agents. The cash dogfood ran, the path was validated, real bugs were captured with reproducible evidence and traceable causal chains. The structural F5/F8/F10 fixes from Phase 9 held up — none of them were the source of any new failure.

Open:

  • Bug 15 (separate plan)
  • useSeatAgent flow-order question (decided in Bug 15 plan)
  • Unfreeze operator endpoint (Bug 15 plan)

Phase 11 readiness: Phase 11 (SnG dogfood) is informally satisfied by the same week's tournament smokes (90 clean hands with zero anomalies after the bookkeeping fixes landed — see SnG Tournaments closed). A formal Phase 11 run waits until Bug 15 is fixed — running a 9-agent SnG dogfood today would hit the same race + freeze.

Now-current pages ​

Sequencing after this ​

The next named work is bulletproof persistence. Bug 15 investigation runs in parallel since it touches admin-server + game-server's seat handler while persistence touches engine serialization + Redis tier — disjoint surfaces.