Same evidence. Same model. The only thing that changes is how you bucket the bullet points.
A language model isn't just weighing facts — it's weighing presentation units. Break one case
into five separate bullet points instead of one merged paragraph, and the model's stated confidence jumps —
by up to +33 points in the paper's experiments — with zero new information added. This mirrors
"support theory" / the unpacking effect long documented in human judgment (list five separate reasons
a bet hits instead of one combined reason, and people rate it more likely). Pick a scenario below, then flip
between Merged and Split and watch the meter move on formatting alone.
Grounded in arXiv:2609.08698 — "Record Grouping Controls Evidence Weight in Language Models"
Scenario
Framing
Underlying facts: 5 — identical in both modes
Handicap Pick: Lakers −4.5 vs. Suns
Evidence exactly as fed to the model — only the packaging changes.
AI stated confidence
Precomputed from the paper's reported grouping effect — no live model call.
+0 pts
0%
—
1 combined argument
…— simulated model verdict, canned replay
—
—
—
0
Why this happens: evidence weight isn't additive by fact — it's additive by "record"
The paper's core claim is that a language model doesn't silently sum up the propositions in front of it —
it partly sums up the presentation units it was handed. One merged paragraph reads as one claim, so it gets
weighed roughly like one claim. The same content sliced into five bullets reads as five independently-corroborating
claims, and the model's confidence inflates as if it had gathered more independent evidence — even though the
underlying facts, and their truth value, are byte-for-byte identical.
That's the same bias support theory documents in human handicappers, jurors, and analysts: unpack an argument into
more sub-reasons and people (and now, apparently, LLMs) rate the unpacked case as more likely true than the packed
one. Nothing below this line changed the facts — it only changed how they were grouped.
arXiv:2609.08698 — "Record Grouping Controls Evidence Weight in Language Models"