Card AI overnight research

autonomous · live-updating · https://share.djiang.xyz/nsba-markets/2026-08-07-cardai-overnight-research.html

Overnight card-AI research — beat the humans at Buzzer

Researcher: Claude (autonomous). Started: 2026-08-07 night. Worktree: nsba-cardai

(branch cardai-overnight, isolated from main + David's cards-collection WIP).

Goal

Produce a Buzzer bot that beats strong humans — or prove it's infeasible (variance ceiling). Report

progress to David via Sage share links through the night.

What we know coming in (from the HITL session)

~82% gauntlet, beats Magnus 56%. Sim is FAITHFUL to live (posture census matches).

full per-card posture VECTOR (aggress ace + Safe the weak tail; safeRank≈2.9, aggRank≈1.9) and

play full-game stamina/slot economy; DeepBlue's depth-2 search + GREEDY rollout continuation can't

plan that.

per-card shield ≈ wash; learned strategic-value leaf predicts 84% but plays even (52.8% vs control

@N=200); heuristic sparring bot still loses to DeepBlue+ (36%).

leaf (leaf_mode="learned", value_features.py), shield knob, research/bench.py harness.

Research threads

from real game logs → HumanClone. Decides "is it fixable": does the clone beat sim-DeepBlue+?

whether depth 3/4 now helps (the "greedy tail is why depth fails" hypothesis).

helps vs everyone.

Measurement

research/bench.py: candidate vs {DeepBlue+, Magnus, MegaSanj, human, (HumanClone)}, parity seats,

live config + real pool, win rate ± 95% CI. Rule: N≥150 before believing any edge; report CIs.

Cycle log

Cycle 0 — setup (done)

Worktree + branch + infra recovered (from stash) + bench harness + this plan. Threads T1/T2/T3

dispatched as subagents.

Cycle 1 status (in progress)

running. Subagent PAUSED — resume it via SendMessage once its battery procs finish to collect the

depth-2/3/4 numbers + verdict.

Cycle 1 result — T1 Human Imitation Clone (DONE, decisive)

**Verdict: a faithful human clone does NOT beat sim-DeepBlue+ → live human dominance is

variance + DRAFT, not a fixable in-game posture hole.**

Pure-Python 3-class softmax posture model; HumanClone(MegaSanj) learns only commit_postures.

Magnus 56.7%. (Real gain over the 36% hand-heuristic, still short of a coinflip vs search.)

AGG_BONUS sweep vs MegaSanj: 0.0→60%, 0.6→44%, 1.0→24%, 2.5→6% — more aggression monotonically

LOSES. Real humans are 47% passive / 35% normal / 17% aggressive (passive-HEAVY). The

"out-tempo by aggressing" hypothesis is FALSIFIED. DeepBlue+ is at/above human posture level.

an ELITE-specific clone (Chris/Orca) + Chris's DRAFT are the untested angles. Clone uses

MegaSanj's draft, so it never tested the draft dimension.

the remaining angles. This corroborates cardgame-ai-ceiling ("only untested lever = draft").**

Cycle 1 result — T3 Planning Continuation (DONE, first positive candidate)

Built plandb = DeepBlue+ with a shield+spearhead rollout CONTINUATION (Safe weak tail +

aggress ace in future rounds, via the posture vector) replacing the greedy single-Contest proxy.

66.5% vs MegaSanj, 63.5% vs Magnus. First bot all session to beat DeepBlue+.

Deeper search is strictly WORSE even with a planning continuation. The value is the *quality of

the 1-step continuation model*, not depth (Monte-Carlo variance + proxy bias compound per ply).

variance-capped ceiling. Launching an N=400 confirmation.

Cycle 2 — COURSE CORRECTION (David pushback, valid)

The "variance-capped ceiling / it's just variance" framing is WRONG: a 5-0 sweep at 190-44 (2:1)

CANNOT come from a game capped at ~55% — repeated 2:1 beatdowns falsify it. There IS real skill

headroom DeepBlue+ isn't capturing. Errors to correct: (1) "posture is maxed" rests on a POOLED

MEDIAN clone — says nothing about ELITE (Orca) play; (2) sim-vs-live fidelity was settled too fast.

**The real puzzle = the contradiction itself: sim says DeepBlue+ is strong; live Orca/pine crush it

2:1.** Leading hypotheses: (A) LIVE DeepBlue+ is degraded vs sim (turn-clock/config/fallback) — a

2:1 live gap the sim can't reproduce screams "different bot"; (B) elite DRAFT (clone never tested

draft). Launched T7 — sim-vs-live forensics: inspect the live bot config vs sim, replay Orca's

actual games and compare live-DeepBlue+ moves to sim-DeepBlue+ on the same states, attribute the

184-91 margin by source, and build an Orca-SPECIFIC clone. This is now the priority thread.

Cycle 2 — RETRACTION: plandb was a small-N false positive

plandb-d2 vs DeepBlue+ at N=400: 48.0% ±4.9 (CI 43.1–52.9) — a WASH, leaning slightly BELOW 50.

The N=160 57.5% was a lucky seed. The planning continuation does NOT beat DeepBlue+. Retracting the

earlier "first bot to beat the champion" claim (reported to David + Sage — corrected).

Pattern is now overwhelming: EVERY in-game change washes at proper N — shield, learned value

(52.8%), planning continuation (48%), pooled imitation clone (loses). Nothing moves the in-game sim

needle. Combined with humans crushing LIVE DeepBlue+ 2:1, the evidence says the answer is NOT in-game

sim policy. It's (A) live-vs-sim divergence [T7] or (B) the draft [T4/T7]. Lesson: these effects are

tiny; require N≥400 before any claim. Doubling down on T7 (forensics) + T4 (draft).

Cycle 2 result — T2 MCTS (DONE, negative)

mcts (UCT over the real engine, posture-vector action set, win-prob leaf) LOSES to DeepBlue+

28.7% ±14 (N=40) at 7.5–10.7 s/game vs DeepBlue+'s 3 s — beaten badly even before compute.

Ties MegaSanj (~53%) like DeepBlue+ does. Insight: in a 1v1 buzz race, Safe-heavy shield lines

CEDE tossups; DeepBlue+ keeps four cards buzzing Normal and out-tempos it. Confirms the posture

ceiling. Artifact: research/mcts.py.

In-game search is now conclusively exhausted (expectimax / value net / CFR / planning-continuation

/ MCTS all wash-to-negative). Remaining levers: DRAFT (T4) and live-vs-sim divergence (T7).

Cycle 2 result — T4 Draft: SYNERGY-STACK is the first promising DRAFT lever (N=400 pending)

draftdb(force_team='Cryofornia Gurls'): instead of MegaSanj's prior-greedy free-pick draft, draft

4 cards of ONE real team across the rarity gates and field all 4 to trigger the engine's TEAM-SYNERGY

bonus (+30 effective speed each at a 4-stack); this team covers all 5 categories with high peaks.

In-game search = DeepBlue+'s, untouched → isolates the DRAFT.

Other objective sweeps (ev/syn/sanity) also pending.

behavior (humans draft synergy teams — cf. specialist/synergy-ceiling memory). If N=400 confirms

>50 with CI excluding 50, it's the first real edge over DeepBlue+. HOLD Sage ping until N=400.

Cycle 2 result — T7 Sim-vs-Live Forensics: THE ANSWER (decisive)

Why elite humans crush DeepBlue+ = knowledge-conditioned aggression the sim can't represent.

NOT variance, NOT a live-bot bug, NOT the draft.

determinizations=5, no timeout/fallback; live seats built make_policy("DeepBlue+") no kwargs;

boost() runs synchronously; turn_timer gates humans only). Hypothesis A (degraded live bot) FALSE.

DB+ slightly faster 60.8 vs 56.7). Hypothesis B FALSE (for Orca's games).

buzzes earlier, clears first, locks DB+ out (48 attempts in 120 rounds), and negs LESS (13% vs 21%).

Converts 87% of attempts across ALL buckets.

23.8%. Both lose in sim; real Orca (same style) crushes byte-identical live DB+ 2:1.

quizbowler who buzzes early AND correctly." It scores blanket aggression as −EV (correctly, for a

knowledge-LESS agent). So DeepBlue+ plays the line optimal vs the sim's model (6% aggression,

disciplined) and gets out-tempo'd by a real expert whose INFORMED aggression converts at 87%. The

sim turns Orca's live 2:1 tossup edge into a 0.88:1 sim DISadvantage. This is why EVERY in-game

method washed — they all optimize against the same miscalibrated conversion model.

A card-stat bot has no per-question knowledge signal, so its aggression negs where the human's

converts → **the edge is largely UN-CLOSABLE for a bot within this engine. DeepBlue+ is the

practical ceiling for a knowledge-less agent.** Only levers: (1) recalibrate the engine's

conversion model vs real logs so skilled early aggression isn't over-penalized (a GAME-BALANCE

change, and it still can't give the bot per-question knowledge); (2) feed the bot a real

per-question knowledge signal (out of scope of the card-stat abstraction).

FINAL REPORT — Can a bot beat these humans at Buzzer? (morning 2026-08-08)

The answer: No — not within this engine. And we now know exactly why.

The elite-human edge (Orca 5-0, avg 184-92; pine too) is **real quizbowl knowledge expressed as

knowledge-conditioned aggression** — buzzing EARLY on questions they know and CONVERTING (~87%

across all difficulty buckets). The game's card-stat conversion/buzz model has **no dimension for

"a player who buzzes early AND correctly."** So a knowledge-LESS bot that aggresses simply NEGS

where a human converts. **DeepBlue+ is the practical ceiling for a bot without real per-question

knowledge**, and the elite-human gap is largely UN-CLOSABLE within the card-stat abstraction.

How we know (T7 forensics — the decisive thread)

determinizations=5, no timeout/fallback). Not a degraded live bot.

the edge.

6%, buzzes earlier, locks DB+ out, negs LESS (13% vs 21%), converts 87%.

sim-DeepBlue+, while real Orca crushes the byte-identical LIVE bot 2:1. The sim turns Orca's live

2.05:1 tossup edge into a 0.88:1 sim disadvantage. → conversion-model fidelity gap, not a bug.

What washed — in-game policy is a proven dead end (N≥400 or decisive)

→ DeepBlue+ is optimal against the sim's model; more in-game search/posture doesn't help.

One curiosity — a DeepBlue+ EXPLOIT, not a strategy

Synergy-stacking (draft 4 same-team cards for the engine's +30-eff-speed 4-stack) beats DeepBlue+

~62% (N=120; **N=400 confirm UNCONFIRMED — still running, and given tonight's false-positive record,

treat as unproven**) — BUT loses to MegaSanj 37.5%, and 28/29 teams are net-negative. → it's an

adversarial exploit of DeepBlue+'s uniform hidden-bench determinization (a team-stacked deck is

out-of-distribution for its rollout), not a strong deck. Value: a KNOWN WEAKNESS of DeepBlue+ to

patch, not a route to superhuman play.

Takeaways for the game (David)

1. DeepBlue+ is a good, honest ceiling bot for knowledge-less play — and humans SHOULD beat it.

That's healthy: it keeps skilled humans winning. "Humans slap the bot" is the game working.

2. To make bots tougher/more human-like: recalibrate the engine's conversion model so skilled

early aggression isn't over-penalized (would push bots toward human tempo) — a balance change,

test carefully; it still can't give the bot real knowledge.

3. Patch the determinization blind spot (team-stacked decks) if you don't want DeepBlue+

exploitable that way.

4. Reusable infra built: per-card posture vector (commit_postures), value pipeline

(value_features.py + trainer), imitation clones (pooled + per-player), bench harness

(research/bench.py), draft variants. All on the cardai-overnight worktree branch.

Honest self-assessment

Two false positives caught and retracted (plandb, and my own over-eager small-N reporting). The

N≥400 rule was learned the hard way. The variance-ceiling framing was wrong and David rightly called

it out; the truth is subtler and better — it's a *knowledge* gap the sim can't model, not "just

variance." Net: a clear, well-evidenced answer to a question that had been fuzzy for months.

CORRECTION (morning) — the "knowledge gap" verdict is RETRACTED

David called this out and he's right. "Humans win via real quizbowl knowledge the sim can't model"

is a CATEGORY ERROR: Buzzer is a CARD game — the human doesn't answer questions, their CARDS do, via

stats, through the SAME conversion function the bot uses. So Orca's 87% conversion is a RESULT of his

DECISIONS (fielding/postures/picks), not human knowledge. The edge is DECISION SKILL → learnable →

bot-closable. The "un-closable" conclusion was wrong and defeatist.

baseline** — it barely beat random, never captured Orca's decisions. "Clone loses" only proves the

clone was garbage, not that elite decisions can't win.

SELECTIVE, well-targeted aggression, a function of observable state → learnable. DeepBlue+ scores

aggression −EV and never finds it.

sim vs DeepBlue+. Win ~2:1 → H1 decision-skill (capturable, bot CAN win). Win ~50% → H2 sim≠live

engine calibration (aggression +EV live, −EV in our sim). Neither is "knowledge."

miscalibrated sim model — if H2, they're moot.

T8 — DECISIVE REPLAY: it's DECISIONS, bot-closable. "Knowledge gap" fully dead.

Reconstructed all 120 rounds of Orca's 5 real games (0 unreconstructable), ran his EXACT moves vs

DeepBlue+'s exact (frozen) moves through the real resolve_round, resampling only chance nodes, N=300.

(byte-identical — same code runs live & in sim). Per-buzz conversion parity mean|Δ|=0.0085. No

calibration gap possible.

edge is entirely IN-MODEL DECISIONS, not knowledge. His live 5-0 (0.672 tossups) was variance-

inflated: model predicts 63.7 tossups, he got 78; ~8 from buzz-noise suppressing thin leads + ~1.1

SD lucky sample. True edge ~63-72% tossups (~65% games), 5-0 had ~8-12% prob.

knowledge-over-speed fielding on the knowledge-gated Nats buckets; postures ≈ +2pp. NOT the deck

(equal ceilings; he even won using DB+'s own captain).

(his deck) scored 0.630 tossups / 4.15/5 games in replay — but that's vs DeepBlue+'s FROZEN

recorded play, not a re-searching DB+. **Must confirm with a clean fresh head-to-head at N≥300

before claiming a bot beats DeepBlue+.** (Not repeating the plandb small-N/over-claim mistake.)

postures), all inside the sim's model → BOT-CLOSABLE. Confirming the fielding lever fresh now.

T9 + SETTLED CONCLUSION (morning, scrutinized)

T9: knowledge-first fielding is a WASH — 52.5% [46.8,58.2] N=300 vs DeepBlue+ (control DB+ vs DB+

= 47.0%; heavily overlapping → indistinguishable). Mechanism proven + independently verified: the

knowledge-max refield proposes swaps but 246/246 are blocked by the bench-lock (min_field_rounds=3,

engine.py:952) + fatigue → knowfield's field is IDENTICAL to DeepBlue+'s. **T8's "4.15/5" was an

ILLEGAL counterfactual** (replay re-fielded freely each round ignoring bench-lock/fatigue). DeepBlue+

already fields knowledge-first as hard as the rules allow. No separable fielding headroom.

The honest, corrected answer (replaces both the "knowledge gap" AND the "just variance" framings)

1. It's DECISIONS, not knowledge (engine byte-identical prod==sim, md5-verified; Orca's real moves

beat DeepBlue+ *in the same engine*). Last night's "knowledge/un-closable" verdict was a category

error — retracted.

2. Orca's real edge over DeepBlue+ is REAL but SMALL: ~+5pp on tossups → ~65% game-win. That's a

clear favorite (real skill, NOT "just variance") — but modest. The **5-0 / 184-92 blowout was that

~65% edge amplified by a lucky sample** (~8-12% probability). Tossup-rate edges compound into big

game-win and score gaps + high per-game variance, which is why a modest edge looked like a massacre.

3. Source of the edge: ~+2pp postures; the rest is pick-steering + a small residual. **Fielding is

NOT a lever** (bench-lock, verified). Draft isn't it in his games (equal decks). Every in-game

method we built washes because DeepBlue+ is already near the DECISION ceiling too, not just the

search ceiling.

4. Can a bot beat these humans? DeepBlue+ is already ~a 35-point underdog to elite Orca — *close*.

Consistently beating elite humans needs capturing a small, diffuse ~+5pp tossup edge (best remaining

candidate = pick-steering; postures/fielding are maxed). Not "un-closable," but no silver-bullet

lever — the headroom is small and spread thin. The human "domination" was a modest real edge on a

hot streak, not a skill chasm.

Remaining open question (cheap to answer): round-by-round replay diff — WHERE (which specific rounds/

decisions) Orca's frozen play out-scores DeepBlue+'s — to localize the residual ~+3pp beyond postures.