PROGRESS.md

Progress & Resume State

Last updated: 2026-05-30 (overnight build). Update this whenever phase status changes.

Status by phase

Phase	Item	Status
0.5	A6 Sports-analytics research	✅ `research/A6_sports_analytics.md`
0.5	A7 ML/stats research	✅ `research/A7_ml_methodology.md`
0.5	A8 Quizbowl prior-art research	✅ `research/A8_prior_art.md`
0.5	A9 CS-strategy research	✅ `research/A9_cs_strategy.md`
0.5	A5 Discord intelligence	✅ `research/A5_discord_intel.md` + `discord_player_signals.csv` (→ D12, INTEL.md)
0.5	scibowl.live coverage/celerity extract	✅ `findings/scibowl_coverage_baseline.md` (→ F1 in KEY_FINDINGS)
0	Foundation workflow (parse + identity + audit)	✅ `wf_b2960c0c-5cb` — audit verdict TRUSTWORTHY, no blockers (`findings/00_data_audit.md`)
1	Core modeling (spine + A10–A20 + 2 adversaries)	✅ `wf_cd35683e-88e` — 9/11 findings trustworthy. Red-teams: `findings/90,91`. F17 (CS 3.5×) magnitude NOT reproduced → re-deriving; F13/F19 minor slips; growth/coverage = nsba3-confounded → revising.
1b	Revisions + draft analyses + synthesis	✅ `wf_9febbe4e-6a6` — `KEY_FINDINGS.md` + `SYNTHESIS.md` written. CS re-derived (~2×→1.2-1.4×, not 3.5×); coverage design-rebuild null; growth net-of-reversion +0.41z; ADP edge ~13 pts/slot; pick chart steeply convex.
2a	Pick-trading strategy + NSBA2 trade back-test	✅ `findings/25_trade_strategy.md` — curve steeply convex; NEVER trade R1 down for volume; David's NSBA2 trade was −2.73 WS (−EV).
—	Thorough `/share` (peer-reviewable)	⏳ PUBLISHING NOW

⚠️ Draft slot "#9" appears in synthesis/OPEN_QUESTIONS as a PLACEHOLDER — NOT confirmed. David learns real slot ~06-06. | 0+ | Historical draft results parse (NSBA1/2/3) | ✅ draft_picks.csv (216), draft_rosters.csv (39), draft_eligibility.csv (152). nsba2/3 100% matched; nsba1 82% (no discord tags). 12/12/14 teams. | | 2 | Draft tooling (A21–A25) + draft-analysis (ADP, pick value, draft→outcome) | ⛔ blocked on Phase 1 + draft parse + NSBA4 pool (finalizes ~06-06) | | — | Synthesis (A30) + thorough /share | ⛔ pending Phase 1 |

Resolved 2026-05-30 (file confusion)

Season mapping confirmed: combine.xlsx = NSBA1 (5 cats, no CS). The "Combine stats and schedule" file = same NSBA1 (richer copy). NSBA2 is aggregate-only in BOTH provided files (no per-question). See DATA_DICTIONARY combine notes.
Modeling guardrails baked into the workflow: filter games reconciles==True (181 clean), subject crosswalk, canonical IDs, combine-de-biking, scibowl F1 baseline.

Queued (David request 05-30): PICK-TRADING STRATEGY

Build on D2's pick_value_chart: (1) price per pick slot & round; (2) talent-concentration / value-cliff shape (steep→trade up for stars, flat→trade down for volume); (3) trade rules reconciling "stars/PPTF win" vs "coverage is table-stakes" (does trading a star pick down for specialists actually pay?); (4) BACK-TEST David's real NSBA2 trade (1st-round pick → 2× 2nd-round picks for specialists): did those specialists out-produce the passed star in realized win-share? Goal = a pre-draft trade plan to acquire high-value picks. "Don't want to RELY on trading, but want the prices + a strategy ready." Apply to David's slot once known (~06-06). → docs/findings/25_trade_strategy.md + outputs/pick_value_chart.csv.

Overnight wave 2 (05-30, post-share)

✅ Preliminary draft strategy (26) — LIVE on share, featured. NSBA2 retro confirms thesis (David's slot-6 trade −2.73 WS); NSBA4 prelim board (Akhil/Rohan/Kian proven; CS via Vishnu M / Harry G).
✅ Bonus value (27): bonuses = 55–61% of scoring (David right on scoreboard) BUT no exploitable PLAYER bonus signal — must win the toss-up to reach the bonus, so PPTF already captures it; knowledge doesn't predict conversion (3 methods agree, coef −0.013 ns). Board barely re-ranks (Spearman 0.95). Nuance: team breadth n_subjects↔win +0.53 (likely talent-confounded — reconcile in resynth).
⏳ Archetypes / duo-synergy / undervaluation-vs-field / field-depth workflow wf_9d1aead9-e46 running (+ skeptic red-team).
✅ Combined RESYNTHESIS done — strategy 26→v2 + KEY_FINDINGS Part C; site republished (47 pages). Duo-synergy NULL, archetypes soft, undervaluation circular (discarded), bonus no-player-signal. Simple strategy survived.

Wave 6 (06-03): LIVE DRAFT TOOL (A24) + ingestion pipeline

✅ scripts/draft_live.py — stateful live draft engine (CLI + importable; state in data/draft_state.json). 14 teams × 6 rounds snake (parameterized), slot set at init (~06-06). Commands: init/pick/me/undo/recommend/board/state/ineligible/captain. Recommends roster-aware marginal value (per-subject θ weighted by MY open/thin subjects, depth bonus for best subject D14, NO CS reach) + VONA "won't last" flag via the field-anchors-on-raw-combine opponent model (F8) + floor/ceiling. Reads joint_value_board.csv + joint_ability_posterior.csv. Default pool = nsba4 combine-takers ∪ intro posters − staff (49 draftable). Snake self-test slot 9 → 9,20,37,48,65,76 ✓.
✅ scripts/refresh.py — one-command ingestion: parse_combine → combine_ability → build_master → A39 intros → reconcile_v2 → worklist → A43 re-fit. ~55s, idempotent, backs up old board, never touches draft_state. Flags: --combine-only/--intros-only/--dry-run.
✅ Intro ingestion fixed — A39 was hardcoded-inline; now also merges data/raw/discord/intros_extra.jsonl AND scans raw intros*.txt to print [A39] NEEDS INTERPRETATION: <handle> for un-structured posts. Clean 2-step path for future drops (drop txt → refresh flags new handles → Claude writes jsonl rows → refresh). Tested.
✅ docs/LIVE_DRAFT.md runbook — data drop locations, refresh usage, draft-day flow (CLI + "just tell Claude"), caveats.
Drop zones: combine → overwrite data/raw/combine/nsba4_2026_combine_INPROGRESS.xlsx; intros → intros*.txt or data/raw/discord/intros*.txt.
⏳ NEEDS DAVID: real draft slot (~06-06); confirm eligible/captain list (use captain/ineligible to prune); resolve the 52-row reconcile worklist (incl. the Rohan G P0485/P0486 duplicate-identity split surfaced by the tool).

Wave 5 (06-03): model-improvement build (joint hierarchical model)

✅ #2 scibowl difficulty anchor → data/processed/scibowl_difficulty_priors.csv + findings/41 (scripts/A41_scibowl_anchor.py). Per-subject difficulty/discrim priors from 13k scibowl buzzes (girth logit scale); question-level b recoverable (373/406 tossups). CS has NO external anchor (scibowl=Energy); 24 scibowl players reconciled to NSBA ids. ESS hardest on scibowl (disagrees w/ F20 → priors are soft, overridable).
✅ #3 backtest scoreboard → scripts/backtest_scoreboard.py (reusable score_ranking(ranking, season)), outputs/backtest_baselines.csv, findings/42. BAR = combine_greedy (nsba2 early-WS 33.2 / ρ 0.49); ceiling = oracle 45.9. nsba2 is the only fully-trustworthy season (57/59 scorable); value_board only ranks n=9.
✅ #1 joint hierarchical Bayesian model (pymc 6.0.1) → scripts/A43_joint_model.py, data/processed/joint_ability_posterior.csv (per player×subject θ mean/sd/80% CI), joint_value_board.csv (288 players: score/floor/ceiling), findings/43. One latent θ[player,subject]; combine + game as two views (combine gets estimated discrim + per-player bias to down-weight biking, D5/D12); scibowl priors on 5 subjects, CS wide. Converges clean (0 divergences, r-hat≤1.006).
⚠️ HONEST RESULT: in-sample it beats the bar (nsba2 early-WS 38.9>33.2, ρ 0.81>0.49, ranks full pool) BUT that's largely same-season leakage — leave-one-season-out it lands SLIGHTLY BELOW raw combine (ρ 0.39/0.32 vs 0.49/0.38). The real, defensible value is the per-subject ability vectors (natively fixes the F40 specialist dilution — William Wang chem +4.26, Kian bio +7.06, Vishnu CS −0.31 confirms fade) + honest posterior uncertainty (floor/ceiling) — NOT a proven OOS predictive gain. n is still the wall. Use joint model for per-subject/coverage-aware decisions + uncertainty; keep raw-combine-anchoring (F8) for slide-exploitation.
⏳ NOT YET DONE from the improvement list: #3-selection/Heckman de-bias of combine→game; #4 draft simulator/optimizer (A22/A25) w/ the field opponent model; recover nsba1 identities + actual nsba3 TUH; republish /share with 41/42/43.

Wave 4 (06-02): intros + generalist-bias audit + strategy v3

✅ Parsed NSBA4 #introductions → data/processed/nsba4_intros.csv + findings/39_nsba4_intro_intel.md (scripts/A39_parse_intros.py). 40 draftable; 34 combine rows, 12 game tape; pool thin-but-better-than-feared. Eligibility/captain notes started.
✅ Red-teamed our OWN generalist bias (David+Anurag pushed). Bias is REAL (PPTF dilutes specialists ~3× in-cat vs all-cat) but localized to mid-board, and undervalued specialists are bio/ESS not CS → refutes D8 "reach for CS." Elite tier unchanged. D14 playoff rule rewards depth in best subject (thin→strip loses 24% vs 9% w/ depth). → data/processed/specialist_adjusted_value.csv, scripts/A_specialist_skeptic.py.
✅ Forensic on the two fades Anurag flagged: Vishnu (oof7373) FADE STANDS (27 games, his "CS" is his worst subject, net −8); Kian (pine2359) DOWNGRADE fade→neutral (genuine bio/chem specialist, top-25 producer, just not #1-overall — don't pay #1 sticker).
✅ Strategy v3 written → findings/40_draft_strategy_v3.md (supersedes 26/v2). Combine-absence steal angle added (David intel: William Wang = buried chem specialist + no combine = double-invisible buy; Sohil Rathi = strong but quant-redundant; Boheng = no tape, confirm).
⏳ NOT YET: republish /share with 39/40; fold corrected lens into a live board once draft slot + eligibility land (~06-06).

Wave 3 (05-30): usage / empty-stats bias (David's friends flagged)

Concern: PPTF/win-shares over-credit a lone star on a WEAK team (shared-toss-up denominator → usage inflation); deflate good players on stacked teams. The "draft the empty-stats guy, he regresses" trap. Assessed as a REAL issue (threatens player rankings, not team-level validation).
⏳ Workflow wf_056ad670-ee8: measure bias magnitude + build teammate/opponent-adjusted (contestation) ability + re-rank + skeptic (does adjustment predict out-of-sample better than raw PPTF?).
NEXT: if bias is material, fold adjusted ability into the value board + strategy v3 → republish. Could shift the buy-low/steal names (weak-team vacuums).

Next actions (in order)

When Foundation workflow completes → review docs/findings/00_data_audit.md (human checkpoint). If audit verdict is trustworthy, proceed; else fix parsers and re-run (scriptPath + resumeFromRunId: wf_b2960c0c-5cb).
Launch the modeling workflow (A10–A20 with A27/A28 adversaries) reading data/processed/.
Launch draft tooling (A21–A25) + A30 synthesis.
Produce a thorough /share brief (what we did + outcomes, peer-reviewable) and finalize OPEN_QUESTIONS.md.

Resume notes

Python: .venv/bin/python. Raw data read-only under data/raw/. Processed tables → data/processed/.
Foundation workflow script: ~/.claude/projects/.../workflows/scripts/nsba-foundation-wf_b2960c0c-5cb.js.
Research briefs already converge on: hierarchical-Bayes IRT (GPCM/MIRT) core; Marcel baseline; VORP/VONA draft value; celerity = combine-only; trust games over combine; CS = scarce specialist (falsify first).
Live intel from David's Discord paste is logged in docs/INTEL.md — includes a testable roster-composition hypothesis and an opponent-GM model.

NSBA Draft Analytics · embargoed until after the SSB draft · ← hub