Fair-Value Model — How It Works & How We Trained It

kalshi-mm • 2026-06-04 • all briefs

What it does (one line)

Given a live chess position + both clocks + both player ratings, it outputs a calibrated probability that each player wins the *encounter* — the number we compare against the Kalshi market mid to find mispricings to trade.

How it works (inference pipeline)


live board  +  clocks  +  ratings
        |
        v
1. STOCKFISH  (pinned: depth 12, multipv 2, 1 thread)
        -> cp        centipawn eval, White POV
        -> best_gap  eval(best move) - eval(2nd) = "only-move-ness" / sharpness
        |
        v
2. FEATURES  (the human-practical "asterisk")
        cp, best_gap, log(white_clock), log(black_clock), white_to_move,
        elo_diff, low-time flags, ply, total_material (phase), queens_on,
        + interactions: cp x time-trouble, cp x rating
        |
        v
3. LOGISTIC MODEL  ->  P(White win), P(Draw), P(Black win)   [classical game]
        |
        v
4. ENCOUNTER COMBINE  (Norway: a drawn classical -> Armageddon ~ coin flip)
        P(player wins encounter) = P(their win) + P(draw) x 0.5

Inference is a microsecond matrix multiply once Stockfish has evaluated the position — the engine eval is the only real cost.

Why not just use the engine's eval?

Raw Stockfish assumes *best play by both sides* — it doesn't know a human with 30 seconds left usually collapses. Our model learns the human-practical mapping from real game outcomes. Concretely: a roughly equal position (eval ~+0.3) but White down to 30s on the clock → the model drops White's win-prob from 52% to 13%. Raw Stockfish still calls it equal. *That gap is the edge.*

In fact, on our data raw Stockfish WDL is worse than just guessing the base rate for elite humans (skill −1.0) — it's overconfident about converting edges that grandmasters routinely draw. So a learned, human-calibrated layer isn't optional; it's the whole point.

How we trained it

Data. ~1,503 games / 20,002 positions from 2026 Lichess broadcasts that include clock data (Norway Chess + several opens, spanning a wide rating range so the model can learn the rating effect). Classical only — Armageddon excluded (different time control). Each sampled position records the Stockfish features + both clocks + both ratings + the final game result as the label (White POV).

Train/live parity. The Stockfish config is pinned and identical for training and live inference (same version, depth, multipv, threads). Training on deep evals but serving shallow ones is the most common way these models silently break — we avoid it by construction.

Model choice. Multinomial logistic regression. GBT gets a marginally better log-loss at this data size but is miscalibrated — and our band strategy needs accurate *probabilities*, not just discrimination — so we ship the verified-calibrated logistic. (Isotonic calibration on top was tried and *hurt* — overfit.)

Validation — blind, by game:

  • GroupKFold by game — the model is always scored on games it *never trained on* (positions within one game are correlated, so we split at the game level).
  • Skill vs base rate: +0.37 (log-loss 0.69 vs 1.09). Adding clock + elo + sharpness nearly doubled the skill over eval-only (+0.19) — direct evidence the human features matter.
  • Calibration ("is 70% actually 70%?") via a blind reliability test: Expected Calibration Error ~1.4% for White-wins, ~3% for Draw / Black-wins. So we trust big disagreements, not small ones.

    • Honest limitations

  • 1,503 games now (up from 70) — probabilities are well calibrated. Skill is slightly lower than the 684-game model (+0.37 vs +0.40) because the bigger set is more diverse/harder = more robust. More always helps, and transfer to a different field (Sinquefield) is still unproven.
  • Trust edges ≥ ~10¢; smaller "edges" can be calibration noise.
  • Trained on Norway + opens — transfer to a different field (e.g. Sinquefield) is unproven.
  • The encounter combine uses a coin-flip Armageddon approximation; a real Armageddon sub-model is still TODO.

    • The files

    engine_eval.py (pinned Stockfish) · features.py (feature vector, shared by train + live = parity) · build_dataset_big.py (data builder) · model.py (training + validation) · fair_value.py (live inference) · calibration_test.py (the blind reliability test) · winprob_model.pkl (the saved model).