Cool Findings โ the highlights reel
Cool Findings โ the highlights reel
A curated tour of the most surprising results, for people who actually know quiz bowl / science bowl. All of it is real code on 3 seasons of game data + 4 combines + 216 draft picks. Small samples (41 team-seasons), so every number is ยฑa lot โ trust the directions, not the decimals. Full detail behind each link.
๐ The headline: one humble stat beats everything we threw at it
We tried five clever ways to build a "better" player rating โ and all five collapsed back to a dead-simple production rate (PPTF = net toss-up points per toss-up faced):
- buzzpoints/celerity, buzz-rank, difficulty-weighting, teammate/usage adjustment, and "cleanup gets" โ each should help, each is a statistical wash out-of-sample.
- The reason: "who converts toss-ups against real opponents, net of negs" already contains nearly all the signal. Everything fancy reduces to it. โ Key Findings
๐ฌ The buzzpoints debate โ settled on real pyramidal data (for Radius)
- We tested whether buzzpoints make a markedly better ranking, on real ACF pyramidal buzz curves (91% of buzzes), not a proxy.
- Plot twist: dan's own Plackett-Luce model uses buzz position not at all for ranking โ the load-bearing ingredient is opponent-adjustment, a model, not buzzpoints. Adding buzzpoints moved the ranking by ฯ=0.978 (โ nothing).
- thedoge's "buzz-rank > celerity" bet loses โ and badly: rank throws away conversion volume, so it buries elite high-volume players (one player: 112 gets, rank 9 โ 319). Volume isn't inflation โ it's the signal. โ 35 Plackett-Luce ยท 36 buzz-rank vs difficulty
๐ธ The draft market is genuinely beatable
- The field drafts the raw combine almost mechanically (ฯโ0.7), but the combine only predicts real value at ฯโ0.4 โ leaving ~13 toss-points per slot of edge for anyone who drafts on projected value instead.
- And the combine is gamed: under "โ1 only for a wrong multiple-choice," late-MC guessing ("biking") is +EV โ so combine rank partly measures guessing strategy, not knowledge. โ 21 ADP / market
- (Sorry Anurag โ drafting on vibe is precisely the inefficiency a value model eats. ๐)
๐งฉ Roster-construction myths, busted
- Coverage is table-stakes: drafted teams already cover all 6 subjects ~96% of the time โ "don't punt a subject" doesn't survive a proper roster-design test.
- Team-building is additive: no duo/chemistry synergy survives (0 of 8 tests) โ so you draft the best player, not the best fit. There is no real "if you have A and B, take C."
- CS is worth somewhat more โ ~2ร, NOT the 3.5ร a first pass claimed (our own red-team caught it sneaking team bonus points into a per-player stat). CS is scarce mostly because nobody studies it โ no canon.
- Bonuses are 55โ61% of all points โ yet carry no per-player signal, because you must win the toss-up to even reach the bonus. โ 16b coverage ยท 29 duo synergy ยท 17b CS ยท 27 bonus
๐งช We put Gideon's roster theory to the test
Gideon's grade-by-role meta โ "ESS main = soph, captain = senior phys/math, chem main = junior quick-with-numbers, bio with some astro" โ we tested it against the data. Verdict: largely unsupported at this sample size โ the specific grade-by-role pairings don't beat just taking the best available players (which tracks with the "additive, no synergy" result above). Still a great falsifiable hypothesis โ more than most "metas" offer. โ 14b growth/aging ยท 28 archetypes
๐ The pick-value curve + an on-brand self-own
- Pick value is steeply convex (Jimmy-Johnson shaped): a Round-1 pick is worth far more than two 2nds โ never trade your R1 down for volume.
- Exhibit A, from our own GM: in NSBA2, David traded his R1 (slot 6) for two 2nds to load up on specialists โ gave up Daniel Sun (a star), netted โ2.73 win-shares, finished 9thโฆ while the team that took Daniel Sun with that pick finished 2nd. The convex curve, in one trade. โ 22 pick value ยท 25 trade strategy
๐ต๏ธ Steals the field slept on
- thedoge โ 2nd-round pick (16th overall) โ rank 3 of 66 in NSBA2.
- Vishwa "Vish" Akkati โ the single biggest NSBA1 steal (pick 33 โ #3 overall in win-shares).
- The steals consistently lived in rounds 2โ5, not round 1. โ 26 NSBA2 retro ยท 34 NSBA1
๐ The honest part (why you can trust the rest)
Small n โ 41 team-seasons โ so everything is ยฑa lot. We ran adversarial red-teams against our own findings and killed several (the original CS "3.5ร", a couple of buy-low flags, a circular undervaluation metric). The buy-low/fade boards are watchlists, not gospel. Trust directions and rankings, not point estimates.
Full project: the hub ยท embargoed until after the SSB draft.