43_joint_hierarchical_model.md

43 — Joint Hierarchical Bayesian Value Model (the centerpiece)

43 — Joint Hierarchical Bayesian Value Model (the centerpiece)

Date: 2026-06-03. Replaces the stitched pipeline (per-season combine GRM in combine_ability.py + separate game-PPTF rate stats) with one joint hierarchical Bayesian model that shares a single per-(player, subject) latent ability vector across two likelihoods. Script: scripts/A43_joint_model.py (pymc 6.0.1, NUTS). Backtest: scripts/A43_backtest.py. Outputs: joint_ability_posterior.csv, joint_value_board.csv, outputs/A43_backtest_comparison.csv.

Model structure (what we settled on)

One latent, two views, hierarchical partial pooling.

• Latent ability theta[player, subject] (6 subjects: bio, chem, cs, ess, math, phys) = player_overall[p] + subject_pop[s] + subdev[p,s], where subdev is partially pooled (tau_subdev). A returning player is ONE latent entity pooled across seasons (no per-season drift latent — n forbids it).
• Game view (outcome currency, D7): per-(player, subject, season) PPTF (net toss-points / tossups-faced) ~ Normal(game_base + game_scale·(theta − subj_diff[s]), game_sigma0/√(weighted TUH)). Precision scales with TUH (a 4-TU subject-season is noisier than a 60-TU one). D19 honored by up-weighting nsba3 game obs (×1.5 vs nsba2 ×1.0, nsba1 ×0.7).
• Combine view (gamed predictor, D5/D12): per-(player, subject, season) combine category points ~ Normal(combine_intercept[s] + combine_discrim[s]·theta + combine_bias[p], combine_sigma[s]). combine_discrim is estimated (the model learns how much the combine loads on true ability) and combine_bias[p] is a partially-pooled per-player offset capturing systematic combine inflation ("biking"). This automatically down-weights a gamed combine relative to game tape.
• Subject difficulty priors: informative N(mean, sd) from scibowl_difficulty_priors.csv (A41) on the 5 anchored subjects; computer_science gets a wide N(0, 1) sentinel prior (no external anchor) so internal NSBA CS data drives it.

Pragmatic reductions (documented, by design): combine observation is the per-(player,subject) category TOTAL (exists for all 4 seasons; the question-level GPCM misses nsba2 and is far heavier), and the game view is a TUH-weighted Normal rate rather than per-buzz IRT. Both keep the structure joint + hierarchical + honest uncertainty — the requirement — without exotic structure n can't support.

Convergence — clean

• 0 divergences, max r-hat 1.006, min ESS_bulk ≈ 860 (full model, 1500 tune / 1500 draws × 4 chains, target_accept 0.95). Samples in ~5 s.
• The only marginally-identified parameter is tau_bias (the combine per-player bias scale) — expected, since a per-player combine offset competes with player_overall; it still mixes fine (ESS ~860, r-hat 1.00).

What the model learned

(1) Combine is a modest, noisy view of ability. Estimated combine discrimination is small and roughly uniform across subjects (0.53–0.90 combine points per unit theta), with physics lowest (0.53) — the combine separates physics ability worst, consistent with F3 (physics weakest combine→game transfer). The model down-weights the combine exactly as D5/D7/D12 demand: game PPTF dominates the latent for players with tape; the combine only anchors those without it.

(2) Subject difficulty: the scibowl prior held; NSBA did NOT override it. Posterior subject-difficulty offsets sit essentially on top of the A41 prior means (bio +0.05 vs prior +0.07; chem −0.13 vs −0.16; ess +0.29 vs +0.32; math −0.13 vs −0.13; phys −0.08 vs −0.10). NSBA game data is too thin to move a well-anchored informative prior — which is the intended behavior: the anchor carries the subjects where internal data is weak. ESS remains the hardest of the anchored subjects; chem/math the easiest.

(3) CS handling worked. With only the wide sentinel prior, CS difficulty moved internally to +0.23 ± 0.51 (from the 0±1 prior) — driven entirely by NSBA CS data, with the posterior SD honestly halving the prior width but staying the widest of the six (we have far less CS signal). CS combine discrimination (0.80) is mid-pack; no CS-specific pathology.

Per-subject re-rating of the marquee specialists (the finding-40 test)

The per-subject vector fixes the scalar-PPTF dilution that finding 40 flagged — it isolates where a player is elite while the projected overall value honestly reflects that they still face all 6 subjects.

This is the core qualitative win: a player is a 6-dim vector, not a scalar. The model rates William Wang's chemistry as elite and Vishnu's CS as poor — distinctions a single PPTF number erases.

Forward nsba4 projection (the live board)

All 288 unique canonical players are on joint_value_board.csv, including 34 nsba4 combine-only players (no game tape). Their 80% intervals are ~1.7× wider than game-having players (mean ceiling−floor 0.454 vs 0.270) — honest uncertainty for players whose only signal is the gamed combine, which is the headline deliverable over the old point-estimate+bootstrap stack.

VALIDATION — the scoreboard (mandatory)

score_ranking on the joint board, vs baselines (outputs/backtest_baselines.csv):

BAR: PASSED. On nsba2 the joint model clears the bar on both axes — early_ws 38.9 > 33.2 and rho_ws 0.81 > 0.49 — and, unlike value_board_greedy (which only ranks 9 players), it ranks the FULL pool (n_corr = 57). It matches value_board_greedy's rho while covering the whole pool, and closes most of the gap to oracle. On nsba3 it nearly doubles combine_greedy's rho (0.75 vs 0.38) at full coverage.

HONEST CAVEAT — same-season leakage, and the out-of-fold null

The in-sample win is substantially same-season leakage: the historical board is fit on the very games it is scored against. The real test is whether the structure generalizes. A leave-one-season-out refit (drop a season's GAME data — keep combine, the only pre-draft signal — and score that season) is far more sobering:

Out-of-fold, the joint model lands slightly BELOW raw combine on both seasons. This is the truthful result and it is consistent with the project's binding constraint (F3: combine→game r≈0.6 and no better; F8: same-season boards leak; n is tiny). Once you remove a season's own games, the only handle on its players is the combine (plus other-season tape for the ~half who return), and the model collapses back toward the combine signal it is built on — it does not manufacture predictive power that the data does not contain.

Interpretation. The joint model's value is NOT a proven out-of-sample ranking upgrade over raw combine. Its real, defensible contributions are: 1. Honest per-subject posterior uncertainty (floor/ceiling, wide bands for combine-only players) — a genuine upgrade over the point-estimate stack. 2. The per-subject ability vector that natively fixes the specialist dilution (William Wang / Kian / Vishnu re-rate sensibly) — a better-aimed mid-board tool, exactly the F40 use case. 3. A single coherent latent that pools across seasons and the two scoring worlds with the scibowl difficulty anchor, replacing the stitched pipeline.

It is not a license to draft rounds 1–2 off this score over the established board; out-of-fold it is combine-grade. Use it as F40 prescribes: the per-subject vectors and uncertainty bands for mid-board re-ordering and specialist identification, not as a proven round-1 predictive engine.

Claim / Evidence / Confidence / Caveat

• Claim. A joint hierarchical model with a per-subject latent, two likelihoods, an estimated-discrimination gamed-combine view, and the scibowl difficulty anchor (a) samples cleanly, (b) in-sample beats combine_greedy at full pool coverage on both trust seasons, and (c) re-rates the F40 specialists sensibly with honest per-subject uncertainty.
• Evidence. 1165 game obs + 1445 combine obs, 288 players; 0 divergences, r-hat 1.006; nsba2 rho 0.81 / early_ws 38.9 (bars 0.49 / 33.2); per-subject thetas in the table above.
• Confidence. High on convergence and the per-subject qualitative re-rating; Med that it is the right structure; Low that it is an out-of-sample predictive upgrade over raw combine (LOSO says it is not).
• Caveat. Same-season leakage inflates the in-sample scoreboard; out-of-fold it is combine-grade (rho 0.32–0.39). n is tiny (3 game seasons); the combine category-total reduction and TUH-weighted-Normal game reduction are pragmatic; subject-difficulty posteriors are prior-dominated (intended). Trust directions, the per-subject vectors, and the uncertainty bands — not the point ranking as a predictive claim.

NSBA Draft Analytics · embargoed until after the SSB draft · ← hub