A6_sports_analytics.md

A6 — Sports-Analytics Methodologies Portable to the NSBA Draft Engine

A6 — Sports-Analytics Methodologies Portable to the NSBA Draft Engine

What this is. A research brief mapping established sports-analytics methods onto the NSBA drafting problem. NSBA is a science-bowl league run like the NBA: players take a 24-question combine (6 categories × 4 pyramidal questions; scoring +4/+3/+2 for earlier/later correct buzzes, −1 penalty for wrong buzzes), then 14 teams build rosters via a snake draft. We have 3 seasons of game scoresheets (tossup-by-tossup, per-player buzzes tagged by subject, plus bonuses) and 4 seasons of combine data. Many players recur across seasons. Samples are small (dozens of players, ~10 games/season) and the combine is gameable (some players tank deliberately).

For each method below: (1) what it is, (2) how it maps to our data, (3) a formula / implementation sketch, (4) pitfalls given our small/biased samples, (5) priority.

Headline framing. Our core estimation problem is identical to the projection-systems problem in baseball: small noisy samples of a recurring population, where naive observed rates wildly over/under-state true talent. The entire Marcel → stabilization → regression-to-mean → aging stack is directly portable and should be the spine of the engine. The plus-minus/EPA/WAR families are situationally useful overlays. Draft-chart and portfolio theory govern how we use the ratings on draft night.

0. The unit of measurement: "points per tossup faced" (PPTF)

Before any method, define the rate stat everything regresses toward. The natural NSBA analog of "OBP" or "points per possession" is expected points contributed per tossup the player was eligible to buzz on. From a scoresheet, each tossup a player was on the floor for yields one of: early-correct (+4), late-correct (+3/+2), wrong-buzz (−1), or no-buzz (0, but a teammate or opponent may have taken it). Per-subject splits (the 6 categories) are the analog of platoon/pitch-type splits. This rate stat — call it PPTF — and its per-category components are the quantities every projection method below operates on.

1. Projection systems: Marcel, ZiPS, Steamer, PECOTA

(1) Explanation. Projection systems forecast next-season performance. Marcel (Tom Tango) is the deliberately-dumb baseline: weight the last 3 seasons 5/4/3, regress toward league mean by adding a fixed chunk of league-average PAs (~1200 for rate stats; the "regression constant"), then apply a simple age adjustment. Despite its simplicity it performs on par with complex systems and is the standard "you must beat the monkey" benchmark. ZiPS (Szymborski) extends this: weighted multi-year average (8/5/4/3 over 4 years for typical hitters), comparable-player aging, and growth/decline modeling. Steamer (Cross et al.) is a stripped-down regression-based system that empirically won several accuracy bake-offs — confirming simplicity is an asset. PECOTA (Silver/BP) is the heavyweight: nearest-neighbor comparables matched on age/body/position/stats, producing a distribution (percentile bands) rather than a point estimate, with aging curves derived from each player's comparables. Sources: MLB Marcel glossary, B-Ref Marcels, FanGraphs projection rundown, BtBS guide to projection systems, PECOTA on Wikipedia.

(2) Mapping to NSBA. A "season" = one NSBA season of game data; the combine is an extra data source (treat like a minor-league line — predictive but on a different scale, needs a translation factor). Build Marcel-for-NSBA: project each player's next-season PPTF (and per-category PPTF) from up to 3 prior seasons weighted 5/4/3, plus a regression chunk of league-average tossups, plus an age/development bump. The combine enters as either an extra weighted "season" (after MLEs-style rescaling) or as a prior for rookies with no game history.

(3) Formula sketch (Marcel-NSBA).

# per player, per category c
num   = 5*pts_y1[c] + 4*pts_y2[c] + 3*pts_y3[c]
den   = 5*tuf_y1[c] + 4*tuf_y2[c] + 3*tuf_y3[c]        # tuf = tossups faced
proj_rate[c] = (num + R * league_rate[c]) / (den + R)  # R = regression constant (Sec 7)
proj_rate[c] *= age_factor(age)                        # Sec 2
proj_points  = sum_c( proj_rate[c] * expected_tuf[c] )

For rookies with only a combine line, set y1=combine (rescaled) and let R dominate so they sit near league average until game data accrues. Ship Marcel first; only add comparable-based aging (ZiPS/PECOTA style) if we later have enough recurring players to fit neighborhoods.

(4) Pitfalls. With ~10 games/season the per-category denominators are tiny, so the regression constant R does almost all the work — get it right (Sec 7) or projections are garbage. PECOTA-style nearest-neighbor comps are not viable with dozens of players (the neighborhoods are empty); don't attempt them. The combine-to-game translation factor is the single largest modeling risk, compounded by deliberate tanking (Sec on combine bias below).

(5) Priority: MUST-HAVE. Marcel-NSBA is the baseline the whole engine is graded against.

2. Aging / development curves (delta method)

(1) Explanation. The delta method estimates aging by looking only at players who appear in consecutive seasons, computing the year-over-year change in their rate stat, weighting each player-pair by the harmonic mean of the two seasons' PAs, and chaining the deltas across age buckets to form a curve. Its known flaw is survivor bias: players who got lucky-good in year 1 stick around to "decline" in year 2, while lucky-bad players get cut and never record the offsetting rebound — this exaggerates apparent aging, mostly past the extremes. Research finds the bias is modest in the middle of the age range. Sources: Tango/MGL aging study, MGL new aging-curve method, BP delta method revisited, BP survivor bias.

(2) Mapping to NSBA. "Development" here is mostly experience/learning, not physical decline — quiz-bowlers typically improve with reps and may plateau, the opposite shape from MLB. Use our recurring players (the whole reason we keep cross-season IDs) to fit a delta curve of ΔPPTF by years-of-experience (or age, if known). This feeds the age_factor() in Sec 1.

(3) Formula sketch.

for each player in consecutive seasons (yr e, e+1):
    delta   = rate(e+1) - rate(e)
    weight  = harmonic_mean(tuf(e), tuf(e+1))
bucket by experience-year; curve[e] = weighted_mean(delta within bucket e)
age_factor as cumulative product of (1 + curve[e]) anchored at a peak/plateau experience level

(4) Pitfalls. Dozens of players × 3 seasons = very few consecutive pairs per bucket; the curve will be noisy — heavily smooth/regress it, or use a single global "rookie→sophomore improvement" term rather than a full curve. Survivor bias is worse for us because there's likely real attrition (players who bomb don't return). Consider modeling experience as a binary "veteran vs rookie" bump first; graduate to a curve only if data supports it.

(5) Priority: NICE-TO-HAVE. A simple rookie-improvement term is must-have-lite; a full delta curve is nice-to-have given sample size.

3. Replacement level / VORP / WAR

(1) Explanation. Replacement level is the performance of a freely-available fringe player (in MLB, ~.294 team win%). VORP/WAR measure a player's value above that baseline, not above average — because the relevant counterfactual is "what could we get for free," not "the average player." There is no single canonical WAR formula (bWAR/fWAR/WARP all differ) but all share: convert contributions to runs/points above average, add the average-to-replacement gap over the player's playing time, convert to wins. Sources: WAR on Wikipedia, MLB WAR glossary, VORP/value-over-replacement explainer.

(2) Mapping to NSBA — the key insight. For a 14-team draft, replacement level is defined by the draft itself: it's the quality of the best player still available just after the last roster spot is filled — i.e., the waiver/undrafted pool. If rosters are R deep, ~14·R players are drafted; replacement level = the projected PPTF at roughly draft slot 14·R + 1 (the best undrafted player). Every player's draft value = projected points above that replacement PPTF, scaled by expected tossups faced. This is exactly VORP and it's what makes ratings comparable across categories/positions.

(3) Formula sketch.

rank all eligible players by projected season points
replacement_rate = projected PPTF of player at index (14*roster_size)   # first undrafted
VORP_player = (proj_PPTF - replacement_rate) * expected_tuf_player
# if positional/category rosters exist, compute replacement_rate per category

(4) Pitfalls. Replacement level shifts with roster rules (how many players per team actually buzz, bench depth). With a thin player pool, replacement level may be close to average, compressing VORP and reducing its discriminating power. Recompute it for the actual league size/roster format rather than importing a baseball constant.

(5) Priority: MUST-HAVE. VORP is the currency that converts projections into draftable value and underlies the pick chart (Sec 8) and portfolio decisions (Sec 9).

4. Plus-minus family: APM, RAPM

(1) Explanation. Adjusted Plus-Minus (APM) regresses point margin on indicators for who was on the floor, isolating each player's marginal impact controlling for teammates and opponents. Raw APM is wildly noisy (multicollinearity from players who always share the floor). RAPM fixes this with ridge regression: an L2 penalty λ shrinks coefficients toward zero, trading a little bias for a large variance reduction — equivalent to a Bayesian prior that players are league-average until proven otherwise. λ is tuned by cross-validation. Sources: Squared Statistics RAPM deep dive I, CMU SCORE RAPM intro, NBAstuffer RAPM.

(2) Mapping to NSBA. Applicable only if scoresheets record who was on the floor each tossup (subs are allowed, so lineups change). If so, each tossup is a "possession": response variable = points swing on that tossup (your team's buzz value minus opponent's), design matrix = +1 for each of your on-floor players, −1 for each opponent on-floor. Ridge-regress to get a per-player RAPM = context-adjusted points-per-tossup impact. This is the only method here that disentangles a player whose stats look good mainly because a strong teammate vacuumed up the easy tossups (or vice-versa).

(3) Formula sketch.

rows = tossups; y = signed point outcome of the tossup
X[row, p] = +1 if p on our floor, -1 if p on opp floor, 0 otherwise
beta = argmin ||y - X beta||^2 + lambda * ||beta||^2   # ridge; lambda via k-fold CV
# beta_p is player p's RAPM (points/tossup above average, context-adjusted)

(4) Pitfalls. RAPM is data-hungry — NBA RAPM needs multiple seasons of thousands of possessions to stabilize; we have ~10 games/season. Expect λ to be large and coefficients heavily shrunk toward zero, meaning RAPM will mostly recover the prior (≈ box-score rating) and add little. Also, in quiz bowl only one player buzzes per tossup, so the "lineup interaction" structure is weaker than basketball — much of the signal is already in individual buzz attribution. Verify lineup data exists before investing here.

(5) Priority: NICE-TO-HAVE (conditional). Worth a prototype iff on-floor data exists; otherwise SKIP. Use it as a diagnostic cross-check on box-score ratings, not the primary rating.

5. Win Shares / box-score value attribution

(1) Explanation. Win Shares starts from actual team wins and distributes them to players via box-score-derived marginal offense/defense, so a team's player WS sum to its win total. Offensive WS are trusted (box score captures offense well); defensive WS are weak because team defense is hard to allocate to individuals. Sources: B-Ref Win Shares, CBB Win Shares calc.

(2) Mapping to NSBA. A "win-shares" approach would credit each NSBA game's outcome to players by their share of marginal points produced (buzzes) and prevented (denying opponents tossups — though "defense" in quiz bowl is mostly just out-buzzing, already in the box score). Because nearly all NSBA value is offensive and individually attributable (you know exactly who buzzed and for how many points), the hard part of Win Shares (defense allocation) mostly vanishes — but so does its advantage over a straightforward points-summed rating.

(3) Formula sketch.

marginal_points_player = points_produced - replacement_rate * tuf_player   # ~ VORP units
team_win_shares = total_team_wins * (marginal_points_player / sum_team_marginal_points)

(4) Pitfalls. Win Shares' top-down "force the sum to equal team wins" step injects team noise into individual estimates — undesirable when we want player talent, not retrodictive credit. With ~10 games the team-win signal is tiny. The bottom-up VORP rating (Sec 3) gives the same information more cleanly.

(5) Priority: SKIP (as a primary method). The framing is subsumed by VORP. Keep only the concept — anchor ratings so they reconcile with observed team results as a sanity check.

6. Expected Points (EPA analog): value of buzzing early

(1) Explanation. Football Expected Points (EP) assigns each game-state (down, distance, field position) an expected net point value; EPA is the change in EP from one state to the next, so each play is graded by how much it improved the scoring outlook. It replaces "all yards are equal" with state-dependent value. Sources: ESPN EP/EPA explainer, Advanced Football Analytics EP.

(2) Mapping to NSBA — strong fit. The pyramidal tossup is a game-state model. Each buzz position within a question has an expected point value; buzzing earlier is riskier (harder clue, higher wrong-buzz/−1 probability) but worth more (+4 vs +3/+2). Build an Expected-Points-per-tossup curve as a function of buzz position and subject, then grade each buzz by EPA = (actual points) − (EP at that buzz position/subject). This directly quantifies the thing the combine scoring rewards: the value of buzzing early, and separates aggressive-but-accurate players from those padding stats on easy late clues.

(3) Formula sketch.

EP(position, subject) = E[points | someone buzzes here]      # fit from all buzzes in corpus
for each buzz: EPA = realized_points - EP(buzz_position, subject)
player_EPA_rate = mean EPA over their buzzes
# blend EPA-rate into the projection as a "buzz quality / aggression value" component

(4) Pitfalls. Per-subject × per-position EP cells are data-thin — smooth across positions (monotone curve) and pool subjects where sparse. EPA rewards aggression, which interacts with the −1 penalty and with combine tanking (a tanker buzzes late/safe on purpose). Use EPA to understand style, but don't let it dominate the talent estimate.

(5) Priority: NICE-TO-HAVE (high value). The single most NSBA-native idea here; great for feature engineering and for valuing early-buzz ability, which raw points obscure.

7. Reliability / stabilization points — our small-sample core

(1) Explanation. A stat "stabilizes" at the sample size where observed results are as predictive as league average — operationalized via split-half reliability. The deep result (Tango): the stabilization point M equals the ratio of within-player (binomial) variance to between-player true-talent variance, and it is exactly the regression constant: the amount of league-average data to add when regressing. Stabilization points are NOT thresholds where a stat becomes "true" — they're the n at which you'd weight observed and league-average 50/50. Sources: FanGraphs randomness/stabilization/regression, FanGraphs reliability update, Probabilaball: stabilization, regression, shrinkage, Bayes, Birnbaum proof of Tango method.

(2) Mapping to NSBA. This is the mathematical answer to our central worry. For each NSBA rate stat (overall PPTF, each per-category rate, buzz-accuracy), empirically estimate its stabilization point M from our data via split-half reliability. M is the regression constant R used in Marcel-NSBA (Sec 1). It tells us, per stat, how many tossups before we trust a player over league average.

(3) Formula sketch.

# Stabilization point (Tango/James-Stein):
M = E[ within-player variance ] / Var( true talent across players )
# Practically, estimate M by the split-half method or by matching observed
# cross-player variance: Var(observed) = Var(true) + E[binom_var]/n  ->  solve for M.

# Regress / shrink any player's rate (identical to Sec 1):
theta_hat = (n/(n+M)) * observed_rate + (M/(n+M)) * league_mean

(See Probabilaball source for the derivation M = E[V(θ)] / Var(θ) and shrink coefficient B = M/(M+n).)

(4) Pitfalls. Estimating M itself needs data — with dozens of players, M's own estimate is uncertain; bootstrap a confidence interval and prefer a conservative (larger) M so we regress more, the right bias under scarcity. Stabilization points are noisy and context-dependent (the sources stress they're "by no means stable"); don't over-trust a single number. Different categories will have very different M (e.g., a deep, consistent subject stabilizes faster than a streaky one).

(5) Priority: MUST-HAVE. This is the rigorous justification for how much to regress and is inseparable from the projection baseline. Build it alongside Sec 1.

8. Draft-pick value charts (Jimmy Johnson, NBA/Pelton, Massey-Thaler "Loser's Curse")

(1) Explanation. The Jimmy Johnson NFL chart (1990s) assigned 3000 pts to pick 1 with a steep decline, derived from observed trade behavior — descriptive, not value-based, and it overvalues top picks ~2×. Massey-Thaler ("The Loser's Curse," 2005) computed surplus value (on-field performance value minus salary cost) and found surplus is flatter and actually peaks in the early 2nd round, not at pick 1 — early picks are overpriced. Pelton's NBA chart values picks by expected career WAR/win-shares at each slot. Common thread: empirical outcome value declines much more gently than naive charts / market prices suggest. Sources: Massey-Thaler "Loser's Curse" / PFF, Open Source Football draft value chart, NFL Ops: building a draft value chart, Pelton NBA pick value (nbasense mirror).

(2) Mapping to NSBA. NSBA picks are tradeable, so we need an NSBA pick-value chart. Build it bottom-up the Massey-Thaler/Pelton way: simulate the snake draft using our projected VORP (Sec 3), and define the value of slot k as the expected VORP of the player actually available at slot k. This yields the empirical decline curve and — crucially — will likely show the same flattening: the gap between pick 1 and pick 14 is smaller than intuition, and mid-snake picks may carry surplus once you account for who's realistically there.

(3) Formula sketch.

run many mock snake drafts (opponents pick by projected points + noise)
chart[k] = E[ VORP of best available player at overall slot k ]
# trade rule: a pick package is fair if sum of chart[k] values is preserved
# surplus view: compare chart[k] (outcome value) to market price of slot k

(4) Pitfalls. Our chart inherits all projection error (Sec 1) — garbage in, garbage out; propagate uncertainty into the chart (use VORP distributions, not point values, so the "Loser's Curse" risk premium is visible). Snake order makes slot value depend on roster already built (positional/category needs), so a single static chart is an approximation — condition on draft context for high-stakes picks. Small pool ⇒ chart is steep early then crashes to ~0 once past replacement level.

(5) Priority: NICE-TO-HAVE. Essential the moment picks are actually traded; until then, VORP ranking alone drives best-player-available.

9. Roster construction / portfolio theory (generalists vs specialists, scarcity)

(1) Explanation. Value-Based Drafting picks the player with the most value over a baseline (VORP = over replacement; VONA = over next-available-at-position; VOLS = over last-starter), explicitly accounting for positional scarcity — the scarcer a position, the bigger your edge from a strong player there. Portfolio framing adds diversification and variance management: balance high-floor generalists against high-ceiling specialists, and value players who cover scarce categories your roster lacks. Sources: FantasyPros VBD (VORP/VOLS/VONA), Subvertadown snake value-based drafting, Advanced NFL Stats: game theory & fantasy draft.

(2) Mapping to NSBA — very strong fit. The 6 categories are positions. A team needs coverage across all 6 subjects (you can't win a science tossup with a literature specialist). So roster construction = covering 6 category "positions", valuing scarcity (if deep science buzzers are rare, a top science player is worth more than his raw VORP), and managing the generalist-vs-specialist tradeoff. A generalist (positive PPTF in all 6) is a flexible "portfolio" asset; a specialist is a concentrated bet with category scarcity value. Use VONA on draft night: pick the category where the drop-off to your next slot is steepest.

(3) Formula sketch.

per category c: replacement_rate[c] (Sec 3); player category-VORP[c] = proj over replacement
team need vector = uncovered/weak categories on current roster
draft score(player) = sum_c category_VORP[c] * scarcity_weight[c] * need_weight[c]
VONA(player) = player_value - E[best available at same primary category at our next slot]
# variance: prefer higher-floor (low projection variance) when leading the draft on value;
#           prefer higher-ceiling specialists when behind / to cover a scarce category

(4) Pitfalls. Scarcity weights are themselves estimated from a thin pool — noisy. Don't over-diversify into mediocrity: 6-category balance is worthless if every player is below replacement. Combine tanking (Sec below) can make a true specialist look like a generalist (uniformly low) or hide a star — lean on game data over combine for category profiles. Correlation matters: category strengths within a player are positively correlated (smart people are broadly good), so the "diversification" benefit is weaker than in finance.

(5) Priority: MUST-HAVE for draft-day logic (it's how ratings become picks); the variance/portfolio refinements are NICE-TO-HAVE.

Cross-cutting concern: the gameable combine (deliberate tanking)

The combine is a biased sample — some players tank. Treatment options, in priority order: - Trust game data over combine wherever both exist; use the combine mainly for rookies. - Detect tanking as an outlier vs. that player's own game-data projection (combine score far below game-implied talent ⇒ down-weight or discard the combine line). - Model the combine as a censored/lower-bounded signal: a high combine is informative (hard to fake upward), a low one is ambiguous (could be tanking or true) — asymmetric weighting. - Never let a single combine drive a projection; the heavy regression constant M (Sec 7) naturally limits combine damage.

Consolidated priority table

Recommended build order: (7+1) regressed Marcel projection → (3) VORP via draft-defined replacement → (9) category-scarcity VBD draft logic → (6) EPA buzz-quality feature → (2) development term → (8) pick chart → (4) RAPM diagnostic if lineup data exists.

NSBA Draft Analytics · embargoed until after the SSB draft · ← hub