Directional Model v1 — Pivot, Backtest & Live Plan
TL;DR
A real pivot tonight, driven by data:
1. Incentive farming is dead at our scale. The per-game books carry ~16,000 contracts resting at the best price (vs a 1,000 target). Our pro-rata share of the $285 pool would be cents-to-single-dollars/day, and the spread is already 0–1¢. Not economic.
2. So we pivoted to a DIRECTIONAL play — *take* a quote when the model strongly disagrees with the market. Deep books actually help here (a taker always fills).
3. Built a much better model. Adding clock + elo + sharpness to the engine eval nearly doubled skill (cp-only +19% → full +34% over base rate). Raw Stockfish WDL stays terrible (−1.0). GBT overfits at this data size.
4. Directional backtest is positive but tiny. Taking the model's big disagreements on Round 8 would've profited — *but it's effectively a 3-game sample.* Promising, not proven.
5. Tomorrow: a $250 directional live test on Round 10, watched.
Why the pivot (the 16k-deep-book finding)
We measured actual resting depth on the Round 8 per-game markets:
| Market | total resting | at best price | incentive target |
|---|---|---|---|
| AFIR | 20,244 | 15,863 | 1,000 |
| VKEY | 26,972 | 18,746 | 1,000 |
| WSO | 23,094 | 18,538 | 1,000 |
At ~$0.018 of incentive per resting contract per day, our affordable size (10–250 contracts) earns single-digit dollars at most, and there's no spread left to capture. Passive market-making these markets isn't viable for us. The edge — if any — is being *right and fast* on mispricings, not providing liquidity.
The better model (v1)
Expanded the data to 70 games / 935 positions (Norway + opens, elo as a feature) and added the human-practical features. GroupKFold by game:
| model | log-loss | skill vs base rate |
|---|---|---|
| raw Stockfish WDL | 2.18 | −1.01 (worse than guessing) |
| cp-only logistic | 0.88 | +0.19 |
| full logistic (cp+clock+elo+sharpness) | 0.72 | +0.34 |
| HistGBT | 1.37 | −0.26 (overfits on 70 games) |
Clock + elo + sharpness nearly double the skill — direct evidence for the "eval with an asterisk" thesis. And the model *behaves* right: a roughly equal position (cp 31) with White down to 30s on the clock → win-prob drops 52% → 13%. Raw Stockfish calls that equal; the model knows a human in time trouble usually loses. That's the kind of moment the market is slow to price.
Directional backtest (the real test — with heavy caveats)
Trained excluding Round 8, then simulated *taking* the model's disagreements with the captured market mids on the 3 R8 games (fees + slippage included):
| edge threshold | trades | hit% | $/contract | $ @ 10-contract size |
|---|---|---|---|---|
| 0.05 | 58 | 57% | +0.13 | +76 |
| 0.08 | 56 | 59% | +0.16 | +87 |
| 0.12 | 48 | 65% | +0.23 | +109 |
| 0.20 | 43 | 65% | +0.23 | +100 |
Reads great — bigger disagreements → higher hit-rate and profit. But be skeptical: those 50+ "trades" are *correlated* bets on the same 3 games (the model called Firouzja-win, Carlsen-loss, So-win — all correct). That's ~3 independent samples. Three coin-flips landing right looks like genius. Also: opens-heavy training vs elite test, and optimistic execution (mid + 1.5¢). Proof of concept, not proof of edge.
Tomorrow's live test (Round 10)
directional_engine.py — for each live game: current board + clocks → model → if it disagrees with the market mid by > 0.12, *take* the underpriced side (small size, crosses to fill into the deep book), hold to resolution.What's built (full stack)
engine_eval (pinned Stockfish) · pgn_features (correct clocks) · features (train/serve parity) · build_dataset_big · model (the v1 trainer) · fair_value (model-based) · directional_backtest · directional_engine (tomorrow's bot) · risk (caps + kill-switch). All dry-run tested.
Honest limitations
Recommendations
1. Run the $250 directional test tomorrow — the cheapest way to learn if the edge is real.
2. Keep expanding training data (more events, multi-year) — the gating dependency.
3. Re-run the directional backtest after each round to grow the sample.
4. Rotate the exposed API key before scaling.