KEY_FINDINGS.md

Key Findings (canonical)

KEY FINDINGS — NSBA Draft Analytics (canonical, A30 synthesis)

Date: 2026-05-30. Lead analyst synthesis (now incl. second wave F27–F31 + skeptic audit F92). This file supersedes the running stub. Each finding gives: the claim (with honest magnitude/range), the evidence + source file, a confidence (High / Med / Low), and the load-bearing caveat. Where a revision (14b/15b/16b/17b) or red-team (90/91/92) changed a number, the revised value is canonical and the original is noted as superseded. PART C (F13–F17) folds in the bonus, duo-synergy, archetype, undervaluation, and field-depth waves, each carrying its F92 skeptic verdict.

Hard sample caps that bound every magnitude below. 41 team-seasons (nsba1=12, nsba2=13, nsba3=16); 207 game player-seasons; 180 clean games; 3 game seasons (nsba1/2/3) + 4 combine seasons; 216 draft picks. CS exists in games only in nsba2/nsba3. nsba3 PPTF uses an estimated-TUH proxy. Read every bolded magnitude as ±a lot; trust directions and rankings, not point estimates.

Two scoring worlds are never mixed: the combine (pyramidal, gameable) is a PREDICTOR only; game value (standard SB, PPTF) is the outcome currency.

PART A — General science-bowl roster-construction findings (embargoed writeup)

F1 — In natural (school) rosters, breadth + speed win decisively

• Claim: The optimal organic roster covers all 6 subjects, runs balanced (low HHI), and buzzes FAST. Concentration/specialization is negatively associated with winning.
• Evidence: SSB-2026 (13,632 buzzes, 84 teams, 366 players, 100% score reconcile). corr(win%, #subjects covered) = +0.52; corr(win%, breadth HHI) = −0.57 (partial −0.30 controlling for total points); corr(win%, avg celerity) = +0.78 (on par with conversion +0.79, just under raw points +0.85). Every top-10 team covers all 6, HHI ≈ 0.18, buzzes ~2× faster than the field. All 6 subjects covered by 90–98% of teams; Biology never fully punted; Energy smallest share (~13.5%) but ~93% covered.
• Source: scibowl_coverage_baseline.md; data/processed/scibowl_*_stats.csv.
• Confidence: High (clean data, large n, 100% reconcile).
• Caveat: Correlational, not causal. Real Science Bowl (Energy, not CS); natural, not drafted, rosters. Celerity is not directly recoverable for NSBA games (buzz timing not logged), so it transfers as a prior, not a measured NSBA quantity.

F2 — Production rate (PPTF) is the north-star value currency

• Claim: Toss-up points per toss-up faced (PPTF) is the best draftable team quantity for predicting standings. How points are spread across a roster (player-HHI, depth, top-2 share) barely matters; total production rate dominates.
• Evidence: corr(PPTF, win%) = +0.624 (raw) / +0.576 SOS-adjusted; positive every season (nsba1 +0.535, nsba2 +0.561, nsba3 +0.752) and survives without nsba3 (+0.50); LOSO out-of-sample R²≈0.26 (spearman 0.53). point_margin correlates higher (+0.80) but is an illegal draft input (mechanical outcome) and is quarantined. PPG_total (+0.73) edges PPTF only by folding in team-answered bonus conversion, which is not per-player attributable. Win-share decomposition is exact (player shares sum to team wins, max abs error 0.0).
• Source: 15_north_star.md, 15b_sos_robustness_revised.md; player_win_shares.csv, team_north_star_features.csv.
• Confidence: High on the ranking (PPTF is the right currency, survives SOS and nsba3-exclusion); Med on magnitudes (n=41).
• Caveat: nsba3 PPTF uses estimated TUH. Use PPTF as the per-player currency, PPG_total only as a team scoreboard check.

F3 — The combine is gameable; raw combine predicts game value at r≈0.6, no better

• Claim: Combine ability (raw, IRT theta, or de-biked) is a moderate, real but noisy predictor of game PPTF — pooled r ≈ 0.62, ridge CV R² ≈ 0.35 (~⅔ of PPTF variance unexplained). Raw combine is the single best linear predictor. De-biking only earns its keep on the heavily-biked nsba1/Energy era (debiked r=0.56 vs raw 0.24); on clean per-question seasons it slightly hurts.
• Evidence: 102–123 linked player-seasons; per-category transfer holds for all 6 subjects (r 0.38–0.54, math/ESS best, physics weakest). IRT validates in-sample (theta↔raw 0.96 where per-question data is rich).
• Source: 10_combine_irt.md, 12_combine_to_game.md; combine_ability.csv.
• Confidence: Med-High on r≈0.6; Med on the de-biking localization.
• Caveat: Selection bias (red-team T5): the linked cohort is players who took the combine and showed up to ≥2 games — it excludes high-combine no-shows/tankers, the exact players who break the link, so r≈0.6 is optimistic for the unproven pool you actually draft. De-biking is partial (question type MC-vs-short-answer is not recorded).

F4 — Coverage is table-stakes the draft auto-provides; "don't leave a hole" is the

only clean breadth effect, and even it does NOT survive a design rebuild - Claim: Drafted NSBA teams reach near-universal 6-subject coverage (95.1% cover all 6; 100% vs 100% at ≥8 games vs natural teams), including CS at 96.6%. Coverage is therefore a floor the draft already clears, not a differentiator. The realized "don't punt a subject" penalty (corr −0.54) does not reproduce when coverage is measured from roster DESIGN (drafted players' combine ability): design-punt → win% corr = −0.108 (p=0.56), null under every threshold and every talent control. - Evidence: Realized coverage corr(win%, #punts) −0.575 on the bridged sample; design coverage −0.108. The two measures barely agree (corr −0.119); only 1 of ~7 punters overlaps. All 5 realized punters are nsba3; SOS-exclusion shows ALL breadth/coverage variance is nsba3-only (nsba1/nsba2 teams all cover 6; subject_hhi→win% = −0.01 without nsba3). - Source: 18_natural_vs_draft.md, 16_generalist_specialist.md (superseded), 16b_coverage_design_revised.md (canonical), 15b_sos_robustness_revised.md, 23_draft_to_outcome.md; team_design_coverage_16b.csv. - Confidence: Med-High that the design effect is null (robust across 8 thresholds, 3 controls, all season slices); the realized penalty is a symptom of weak/short- schedule teams, not a roster-construction lever. - Caveat: Effective test = 24 nsba2/nsba3 team-seasons via an imperfect draft→game bridge. Actionable residue: don't actively punt a subject (the snake draft hands you coverage anyway); breadth is otherwise mostly a proxy for talent (red-team L2).

F5 — CS is worth somewhat more than a generic point, via confound-robust legs —

NOT 3.5×, and the scarcity premium is already arbitraged away - Claim: The original "CS ~3.5× / t=4.55" headline does not survive — it folded team bonus points into a per-player value claim. On toss-up points only, CS≈2.0× generic (t=2.08, p=0.047, bootstrap 95% CI [−0.28, 7.14] — cannot rule out CS being less valuable), collapsing to 1.4× under SOS and 1.2× under team fixed effects. What does hold are player-level legs: CS-mains convert CS 62.0% vs generalists 48.6% (χ²=7.00, p=0.008), and combine theta_cs → in-game CS r=0.60. CS is covered by 96.6% of teams, so coverage is table-stakes; depth/speed is a modest, uncertain tiebreaker. - Source: 17_cs_value.md (superseded), 17b_cs_value_revised.md (canonical), 15b_sos_robustness_revised.md, red-team 90 (H1/M1), 91 (T3). - Confidence: Med on direction (CS worth more, specialist not redundant, combine identifies CS via theta_cs); Low on any magnitude. - Caveat: 29 team-seasons, 2 seasons, 101 combine-linked players, 8-player CS-main group (lean on the 514-buzz pooled χ², not the split). The "premium erodes as adopted" decay test is underpowered because the field already covers CS. Do NOT reach.

F6 — Development is positive but small NET of mean-reversion, and does not transfer to

new entrants - Claim: Returners improve, but the naive "+0.43 z/edition" is partly regression-to- mean of a negatively-selected returning cohort (returners start ~0.18 z below the field on their first combine). Net of reversion: debiked combine theta +0.41 z (p=0.006, 95% CI [+0.13, +0.68]); game PPTF net +0.22 (p=0.028). Reversion inflated the pooled combine number only modestly (raw +0.46 → net +0.41) — it was not mostly reversion. - Source: 14_growth_aging.md (development half superseded), 14b_growth_net_revised.md (canonical), red-team 91 (T1). - Confidence: Low-Med (22–41 transitions; theta is within-season z, so "growth" is relative-rank climb; 24/30 combine transitions are into nsba4 which has no games). - Caveat: Two draft implications are load-bearing: (1) do NOT add a growth bump on top of reversion-shrink in the buy-low board — that double-counts the same climb; use one model proj = prior + β1·prior + β0 (β1≈−0.43, β0≈+0.41). (2) Growth does NOT transfer to a brand-new nsba4 entrant with no prior — price them off shrunk combine theta only.

F7 — Question difficulty varies by season/subject; playoff "difficulty" is confounded

• Claim: nsba3 packets were easiest; CS is the hardest subject (lowest conversion ~70%); playoffs convert lower (OR=0.59). Season/subject decomposition is a clean logit on 4,155 questions and reproduces.
• Source: 20_difficulty.md, red-team 90 (M4), 91 (T4).
• Confidence: Med on direction; Low on the composed multiplicative "park factors."
• Caveat: Playoff factor is nsba3-only (n=265) and conflates harder packets with stronger surviving fields — do not treat 1.11 as a pure packet-difficulty multiplier, and do not multiply the 9 factors without a joint CI.

PART B — NSBA4-specific draft implications

F8 — The field drafts the visible raw combine almost mechanically; ~13 recoverable

toss-points/slot by drafting on projected game value instead - Claim: Draft pick order tracks the raw combine tightly (|ρ| 0.67–0.75 every season — managers anchor on the visible score, not any bias correction). That combine only weakly predicts realized value (ρ ≈ 0.36–0.43), leaving a defensible ~13 toss-points/season/slot (~65 over a 5-round draft) recoverable by drafting on projected game value; oracle ceiling ~35. - Source: 21_adp_market.md; outputs/adp_table.csv. - Confidence: Low-Moderate. The directional results (field anchors on raw combine; combine is a noisy value signal; value-greedy beats combine-greedy) are consistent and significant across the two usable seasons. - Caveat: The honest ex-ante edge number is nsba2 alone (60 picks); nsba3's equal number is circular (same-season leakage). nsba1 nearly blind (11 links). Survivorship makes the combine look more predictive than it is, so the true edge is likely larger.

F9 — Snake-pick value is steeply convex (Jimmy-Johnson shape); the real loser's curse

is variance, not undervalued early picks - Claim: Round 1 retains only 46% of its value into Round 2; rounds 5–6 are nearly flat. No top-end flattening — the curve is convex (a₂=+3.5), so paying linear prices undervalues early picks. The genuine late-pick problem is variance: R1 bust rate 15% vs R6 73% — late picks are lottery tickets. Tradeable chart (pick 1 = 100): R1 100, R2 42, R3 22, R4 15, R5 12, R6 11. - Source: 22_pick_value.md; outputs/pick_value_chart.csv. - Confidence: Low-Moderate. The shape (steep-top, flat-tail) is robust across all three cuts (nsba2-gold, observed-only, toss_points) and is trustworthy. - Caveat: Per-slot values are small-n point estimates with wide bootstrap bands (pick 1: 1.5–3.7 WS). Chart prices slots; subject-coverage/CS fit and the 4–6 picks- per-team rule sit outside it.

F10 — Availability busts are real and common in the most recent format

• Claim: 37% of nsba3 drafted players never played a clean game (vs 0% in nsba2); even 3 of 14 nsba3 captains never fielded. Earlier picks have markedly better availability (pooled spearman −0.36; R1 ≈ 85% vs R6 ≈ 23%). The discord availability_flag tracks realized availability monotonically (none 0.86 / some 0.77 / concern 0.49, but n=16). CS-mains show NO availability penalty (0.71 vs 0.70).
• Source: 24_realized_availability.md; outputs/realized_availability.csv.
• Confidence: Med (directions robust, reproduced from real data).
• Caveat: n tiny (144 trustworthy picks; flag cross-ref on 16); headline bust rate is driven almost entirely by nsba3. Apply reliability_prior to discount part-time assets; for new entrants use the round-conditional base rate.

F11 — The draft "matters" because talent wins, but the combine-seeded draft is a weak

handle on that talent - Claim: Realized roster scoring rate → win% is strong (r ≈ 0.68, clean subset, CI [+0.37, +0.85] excludes 0; star/rate-driven, not depth- or coverage-driven). But the arrow that makes the draft work — combine θ of who you drafted → realized roster PPTF — is weak (r ≈ 0.1–0.3, CIs include 0). Coverage/CS design did NOT separate winners (saturation). The design_has_cs "−0.34" is an n=4 fluke, not evidence CS hurts. - Source: 23_draft_to_outcome.md; draft_outcome_features.csv, draft_team_bridge.csv. - Confidence: Low-Moderate — direction trustworthy, magnitudes not. - Caveat: Clean test rests on nsba2 (n=12) + Jaccard≥0.5 subset (n=19); nsba1 effectively unlinkable (median roster Jaccard 0.11). Bootstrap CIs ~±0.3 wide.

F12 — Buy-low / fade board: a watchlist, not a measurement

• Claim: Players whose game tape beats their combine are buy-low candidates; combine- inflated players (likely bikers) are fades. Game-backed buys: Mihir K (modernnewton, conf 0.71), Rohan G (0.79). Game-backed fades: Vishnu M (oof7373, −1.20, conf 1.00 — most data on the board), Kian D (combine #1 overall but not #1 in games). Intel-only buys (Ziang, nocombomomento) rest on Discord chatter (conf 0.40).
• Source: 19_tanking_buylow.md; outputs/buylow_board.csv. Anchor is raw theta_overall, r=0.74 (corrected from "de-biked", red-team M2; de-biked anchor 0.67).
• Confidence: Low (use for shortlisting, not point estimates).
• Caveat: Track A = 18 players, 1–27 game reads; Track B = intel priors only. Several top buys may be captains (can't be bought). santhosh_b. EXCLUDED (confirmed cheat, combine inflated). Cross with eligibility/captain list before acting.

PART C — Second-wave findings (F27–F31; skeptic-audited in F92)

F13 — Bonuses are 55–61% of the scoreboard, but carry NO exploitable per-player signal

• Claim: Bonuses are the majority of points (gross toss-up basis 0.553, net 0.611; per-season 0.51–0.57) and decide a real minority of games (~15% won on bonuses despite losing the toss-up battle). But book-knowledge does not predict bonus conversion, so there is no hidden per-player bonus value — a knowledgeable-but-slow player is not undervalued by PPTF. You earn bonuses by winning the toss-up, which PPTF already captures.
• Evidence: 181 clean games, 3 methods all non-positive: player-conversion ↔ combine theta +0.016 (n=73); ON/OFF lineup effect −0.034; roster-knowledge → conversion coef −0.013 (p=0.37) / max-theta −0.008 (p=0.57). The 8 "knowledgeable-but-slow" players convert at 0.542 vs league 0.489 (n=8, noise). Full-bonus-credit board re-ranks ≤1 spot vs PPTF (Spearman 0.945 full / 0.996 knowledge-premium-only) — movers are toss-up volume scorers, not slow-smart bodies. Bonus conversion IS a real team skill (~49%, std 0.20).
• Source: 27_bonus_value.md; outputs/bonus_value_summary.csv, value_board_with_bonus.csv, team_knowledge_bonus.csv.
• Confidence: Med-High that there is no per-player bonus signal (3 independent methods, consistent non-positive sign); High on the scoreboard share arithmetic.
• Skeptic verdict (F92): not separately audited; the per-player null is robust across methods. Action: do NOT add a per-player bonus-conversion term; do NOT value slow-but-smart bodies for hidden bonus value. Any bonus adjustment must be team-level only (~±0.2 swing, low-confidence).
• Caveat: Bonus attribution is inherently speculative (team-answered); lineup data exists only for nsba1 + partial nsba2; KBS sample n=8; team-knowledge regression n=35. The team breadth↔win +0.53 is real but likely talent-confounded — same confound that killed the realized-coverage penalty (F4); do not treat breadth as an independent lever. Reproducibility gap: the F27 script (analysis_bonus_value.py) is not committed to scripts/.

F14 — Duo / pair synergy is a clean (bounded) NULL → team-building is ADDITIVE

• Claim: Teammate pairs' joint production is statistically indistinguishable from the additive sum of the two members' individual baselines. Neither archetype-complementarity nor subject-disjointness predicts super-additive production. Draft best individual value; do NOT chase fit, complementarity, or "covers each other" pairs.
• Evidence: 474 co-rostered pairs in 38 team-seasons; all clean inference at team level (n=38) or team-season cluster bootstrap. Complementarity ↔ synergy corr +0.025 (p=0.88); 0 of 8 tests survive BH-FDR. The one nominal raw hit (elite+repl q=0.037) is additive inheritance, not interaction — the decisive test (does a non-elite's own residual rise with an elite teammate?) is +0.135 WS, p=0.155, ns. The disjoint-pair "effect" points the wrong way and is a talent-density confound.
• Source: 29_duo_synergy.md; data/processed/duo_pairs.csv, outputs/duo_archetype_pairs.csv, duo_team_synergy.csv. Script scripts/A29_duo_synergy.py.
• Confidence: Med-High on the null direction (best-engineered finding in the wave).
• Skeptic verdict (F92): TRUST. Pseudo-replication correctly defused, FDR honest, the nominal hit cleanly explained. No overreach found.
• Caveat: Bounded null, not proven-zero — a true synergy smaller than ±0.3 WS/pair would be invisible at n=38. "Solo baseline" is unobservable (defined as residual above an additive talent baseline). Leans on soft F28 archetype labels and combine theta.

F15 — Subject-archetypes are real but SOFT; only the broad elite tier reliably wins; CS is not a type

• Claim: Five soft subject-archetypes emerge (bio / ESS / elite-multi-science / math-phys / replacement), but only the broad "elite multi-science" tier reliably wins (~4× the WS and essentially all VORP of any specialist). Specialists (bio/ESS/math-phys) are middling, interchangeable coverage. CS does NOT form a standalone archetype — high-CS players cluster with the broad elites. Use directions (broad-and-deep wins), not labels.
• Evidence: 344 player-seasons; k=5 by silhouette (0.234 — weak separation, soft blobs). Elite-multi-science PPTF 0.71 / WS 2.5 / VORP 44.6 >> all specialists (~0.3–0.4 PPTF). CS-aware re-cluster (ARI 0.72) puts high-CS with broad elites, no pure-CS pod.
• Source: 28_archetypes.md; data/processed/player_archetypes.csv, outputs/archetype_profiles.csv. Script scripts/A28_archetypes.py.
• Confidence: Low-Med. Trust the direction (a broad-and-deep top tier wins; specialists incl. CS are interchangeable coverage); distrust individual labels.
• Skeptic verdict (F92): WATCHLIST. Stability is overstated by the wrong metric — leave-one-subject-out ARI drops to 0.52 (dropping bio or ESS reshuffles ~half the assignments), and the load-bearing elite cluster has only median 0.64 membership recovery. The direction and the CS-absence claim are safe; individual archetype tags are soft inputs, not facts (and they propagate into F16/F17 below).
• Caveat: n tiny, silhouette 0.23; 137/344 rows are combine-only nsba4 (100% combine, as noisy as the combine). Use as descriptive tendencies, not hard labels.

F16 — "Value-over-field" undervaluation: the new validation is CIRCULAR; no new edge

• Claim: The v2 per-player "value-over-field" metric (uvf_combine = our-PPTF-pct − combine-rank-pct) does NOT add a new undervaluation edge. Its flagship ρ=0.60 "validation" is ~90% a mechanical shared-PPTF artifact. The only real edge remains F8's established ~13 toss-pts/slot (value over raw-combine rank). Keep the forward nsba4 board as a soft watchlist for within-tier re-ordering only.
• Evidence: n=94. Headline ρ(uvf_combine, val_resid)=0.599, but randomizing the combine signal barely moves it (0.54 — survives pure noise); PPTF alone correlates with val_resid more strongly (0.71); and combine's own independent link to the slot-residual is 0.007, p=0.95 (flat null). BH-FDR: 0 of 94 names individually significant.
• Source: 30_undervaluation.md; outputs/undervaluation_board.csv. Script scripts/A30_undervaluation.py.
• Confidence: Discard the ρ=0.60 validation as evidence for F8; Low watchlist on the forward board.
• Skeptic verdict (F92): DISCARD the validation; WATCHLIST the board. "Validates the F8 mechanism per-player" is not supported — do not cite ρ=0.60 as confirming F8. The combine-only forward board is a separate, internally-consistent exploit of "the field anchors on raw combine" — keep it only as a tier re-order device. Santhosh V independently re-lands on the fade list (confirms the cheat exclusion).
• Caveat: Same-season leakage; percentile metric compresses extremes; inherits soft F28 labels; assumes the field still anchors on raw combine this year.

F17 — Field depth under playoff difficulty: lean broad elites (LOW confidence, nsba3-only)

• Claim: Under playoff difficulty, single-subject ESS specialists collapse (within-player retention 0.44) while broad chem+phys elites hold or rise (1.47) — difficulty bites the quantitative/Earth-Space cluster (ESS/Math/CS/Physics) and spares recall (Bio/Chem). Lean to the broader chem/phys-flavored producer for playoff equity. The combine does NOT detect playoff-robustness (predicts regular PPTF better than playoff; retention corr ~0.1, ns).
• Evidence: nsba3 only — 12 clean playoff games, 265 questions, 36–43 players. ESS conversion drops −22% (only subject surviving FDR, q=0.012); ESS retention CI [0.22,0.63] and elite CI [1.16,1.74] both exclude 1.0. Combine→playoff Spearman lower than →regular for every theta variant (debiked weakest, 0.42).
• Source: 31_field_depth.md; data/processed/playoff_robustness_31.csv.
• Confidence: Low — directional, one season.
• Skeptic verdict (F92): WATCHLIST. Two flagged rows survive leave-one-out within nsba3, but the binding limit is n=1 season (CIs resample players, not seasons; elite "+47%" rests on n=5). The per-subject ESS drop and the ESS-specialist collapse are the same fact, not two confirmations. A tiebreaker, not a re-ordering lever.
• Caveat: Difficulty vs surviving-field-strength is unidentified; depth counts are selection artifacts (not a depth result). Reproducibility gap: the F31 script (A31_*.py) is missing from scripts/ — only the artifact exists; partially non-reproducible.

Reconciliation ledger (what changed, and why this file trusts the revision)

Bottom line. The thesis spine — draft for projected production RATE (PPTF/VORP), treat coverage incl. CS as cheap insurance the draft auto-provides, and exploit the field's mechanical raw-combine anchoring — survives every red-team and the second wave firms it up: team-building is additive (no duo-synergy lever, F14/F92-TRUST), bonuses add no per-player signal (F13 — rate already captures them), the broad-elite type is the only one that wins (F15), the "new" undervaluation edge was circular (F16/F92-DISCARD, leaving F8's ~13 pts/slot as the only real edge), and playoff-robustness is a low-confidence lean toward broad elites (F17). The two most aggressive original actions (reach hard for CS; punt-avoidance as a lever) remain demoted to tiebreaker / table-stakes, and three new tempting levers (chase fit, value slow-but-smart bodies for bonuses, a second undervaluation edge) are rejected. Trust directions; distrust magnitudes; do not reach. Reproducibility gaps to close: the F31 and F27 scripts are not committed to scripts/.

NSBA Draft Analytics · embargoed until after the SSB draft · ← hub