Deck-Count Duel

A real tabular Q-learning agent learns blackjack from scratch, and a Hi-Lo card counter plays basic strategy with a count-based bet spread — both against an actual shuffled shoe whose size you control. Every number on this page comes from live simulation running in your browser right now: no pre-baked tables, no fake charts.
arXiv:2308.07329 — "Variations on the Reinforcement Learning performance of Blackjack"
What the paper found: shoe size changes two things at once — (a) how fast a Q-learning agent converges to good strategy, because a smaller shoe depletes and reshuffles more often, shifting the state-visitation distribution the agent trains on; and (b) how much edge a Hi-Lo counter actually realizes, because true count = running count ÷ decks remaining, and more decks in play dilute that signal toward zero. Drag the deck-count slider below — both the Q-learning convergence curve and the counter's realized edge should visibly shift. A continuous shuffle machine (CSM) is included as the limiting case: it reshuffles every hand, so the count carries zero information and the counter's edge should collapse to (or below) the flat house edge.
Shoe size4 decks
Continuous shuffle machine (∞ decks, count = 0 always)
Rules: dealer stands on all 17s, blackjack pays 3:2, double on first two cards allowed for the strategy bot. No splits (simplification, see footer).
idle
Q-agent episodes trained (this shoe): 0
States learned: 0
Counter hands simulated: 0
Current true count (sample): —
Q-agent avg return / hand
—
greedy policy, held-out hands, hit/stand only
Basic strategy (flat bet)
—
no counting, house edge for this shoe
Hi-Lo counter edge
—
basic strategy + true-count bet spread
Counter's edge over flat
—
how much counting is actually worth here

Q-learning convergence

Rolling average return per hand (window of 2000 hands) as the tabular Q-learning agent trains from a blank Q-table against the current shoe. Dashed line = the best a hit/stand-only policy can do (computed from basic strategy with doubling disabled, for a fair ceiling). Faded lines are other deck counts you've already trained, for comparison.

Realized bankroll: flat betting vs. counting

Cumulative profit (in betting units) over a fresh run of simulated hands at the current shoe size, basic strategy throughout — one line bets flat, the other sizes bets off the Hi-Lo true count.

Edge vs. deck count

Realized edge (profit ÷ amount wagered, 900k simulated hands per point) across every shoe size, flat-bet vs. Hi-Lo count-based betting. The marker shows your current slider position. This is the core claim from the paper: the counter's edge erodes as decks increase, and collapses under a continuous shuffle machine.