PLAN.md

Agent Architecture & Plan

Agent Architecture & Research Plan

Phased by dependency. Status tracked in PROGRESS.md. Research briefs in docs/research/.

Phase 0 — Foundation (data; everything depends on it)

• A1 Combine Parser → tidy combine tables (per-question pyramidal points, category, overall).
• A2 Scoresheet/Tossup Parser → tossup-level long tables (standard SB scoring); verify lineup recoverability (gates RAPM).
• A3 Identity Reconciler → canonical player IDs across seasons + combine↔game↔scibowl, with needs_review flags.
• A4 Pool/External Extractor → NSBA4 draftable pool (paid+combined); parse scibowl.live.
• A26 Data-Integrity Adversary → audit the parse (docs/findings/00_data_audit.md).

Phase 0.5 — Parallel intel/research (no data dependency) — DONE

• A5 Discord Intelligence → sentiment, networks, study habits, availability, tanking, grade. → research/A5_discord_intel.md + data/processed/discord_player_signals.csv.
• A6 Sports-Analytics Researcher → Marcel, aging curves, VORP/WAR, RAPM, EPA, stabilization, pick charts. → research/A6_sports_analytics.md.
• A7 ML/Stats Researcher → IRT (GPCM/MIRT), hierarchical Bayes, shrinkage, tanking-as-residual, validation. → research/A7_ml_methodology.md.
• A8 Prior-Art Researcher → existing quizbowl/SB stats (PPB, celerity, powers, QBJ); where we're novel. → research/A8_prior_art.md.
• A9 CS-Strategy Researcher → scarce-specialist draft framework (VONA, tier-counting). → research/A9_cs_strategy.md.

Phase 1 — Core modeling (needs Phase 0 data)

• A10 Combine Ability (IRT) — hierarchical GPCM/MIRT latent ability + question difficulty; pyramidal-aware.
• A11 Reliability/Stabilization — split-half stabilization point M = the regression constant.
• A12 Combine→Game Validation — does combine predict game output? incl. playoff/harder-round test.
• A13 In-Game Player Value — VORP on PPTF (primary); RAPM if lineup data; bonus-conversion as separate skill. (Skip Win Shares as primary per A6.)
• A14 Growth/Aging Curves — multi-season deltas + development term; grade-aware if A5 yields grades.
• A15 North-Star Discovery — which team-level quantity best predicts standings → defines value metric.
• A16 Generalist vs Specialist & Subject Coverage (headline) — win-share of balanced vs specialized; coverage as set-cover + portfolio variance; the optimal coverage profile and how common solid coverage of each subject is (vs scibowl natural-roster baseline); test Gideon's specific complementary pairings (bio+astro, chem+phys) and grade-by-role claims.
• A17 CS-Value Quantification — empirical CS win-share from NSBA2/3; test the A9 thesis (generalist-coverage falsifier first).
• A18 Natural vs Draft Rosters — scibowl.live schools (organic) vs NSBA (drafted) coverage patterns.
• A19 Tanking Detection — combine residual vs game/external/latent ability → buy-low board.
• A20 Difficulty Normalization — "park factors": alumni-vs-packet, RR-vs-playoff difficulty.
• Adversaries: A27 Methodology (overfitting/leakage/confounds), A28 Causal/Selection (tanking/survivorship/home-adv).

Phase 2 — Draft strategy & tooling

• A21 Pick Value & Trade Chart (snake, bottom-up from simulated drafts of projected VORP; expect Loser's-Curse flattening).
• A22 Draft Simulator / Strategy Optimizer (Monte Carlo 14-team snake; BPA vs need; CS reach timing).
• A23 Value Board (projection + floor/ceiling + VORP + category scarcity).
• A24 Live Draft-Day Tool (updates as players come off board; recommends pick given roster needs).
• A25 Roster Optimizer (best 5–7 under coverage/complementarity; home-advantage "remove a pair").
• A29 Conclusion Red-Team (stress-test final recommendations).

Synthesis

• A30 Findings Synthesizer — embargoed general SB roster-construction writeup + NSBA4 draft playbook.

Deliverable infra

• Skills: nsba-data (load tidy tables), draft-board (regenerate board), nsba-sim (run draft sim).
• Notebooks: one per analysis (combine-IRT, generalist-vs-specialist, CS-value, growth, draft-sim).

Orchestration

Deterministic spine (parse → model → adversarially verify → synthesize) runs as Workflows; research agents run standalone. Foundation workflow first (with a human checkpoint on the audit), then the modeling+synthesis workflow.

NSBA Draft Analytics · embargoed until after the SSB draft · ← hub