LLM CALIBRATION CASINO

An LLM never says "I don't know." Every claim comes out with the same confident tone β€” whether it's a textbook fact or a total fabrication. You've got $1,000 in chips. Bet TRUE, FALSE, or UNVERIFIABLE on each claim before the house reveals what a real fact-check found.

~1.3M
facts pulled straight from LLM memory
68.4%
actually true, per LLMPEDIA's audit
30.5%
not even Wikipedia could confirm or deny

House Rules

  • 50 claims, dealt one at a time. Nothing is looked up for you β€” you're auditing raw, unverified assertions, exactly like the LLMPEDIA researchers did to GPT‑5‑mini, DeepSeek‑V3.2 and Llama‑3.3‑70B.
  • Each claim comes with a simulated "model confidence" β€” how sure the LLM sounded stating it. That number is a red herring. The paper's whole point: stated confidence barely tracks truth.
  • Pick a bet type, pick a stake (bigger stake = you're claiming higher confidence), lock it in, see the real verdict.
  • Survive all 50 claims (or go bust trying) and the casino shows you the books: your accuracy and calibration side‑by‑side with the paper's real numbers.
TRUE Β· pays 1.6Γ—most common outcome, low payout
UNVERIFIABLE Β· pays 2.8Γ—the sneaky middle ground
FALSE Β· pays 4.5Γ—rare β€” flagship models don't often flatly lie about well-known things
Inspired by "LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs" (arXiv:2609.01182). All claims below are hand-written for gameplay, not extracted from the paper.